2024-01-25T15:47:17.211 INFO:root:teuthology version: 0.0.1.dev251+gc856e0f 2024-01-25T15:47:17.211 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} --name vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/orig.config.yaml 2024-01-25T15:47:17.272 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-25T15:47:17.379 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549 branch: wip-vshankar-testing-20240125.044502 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7531549' kernel: client: sha1: distro flavor: default kdb: true sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240125.044502 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 661946e7044723130c03627323f6a879a9848cb5 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 661946e7044723130c03627323f6a879a9848cb5 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240125.044502 sha1: 661946e7044723130c03627323f6a879a9848cb5 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9619 sha1: 661946e7044723130c03627323f6a879a9848cb5 sleep_before_teardown: 0 subset: 293/1024 suite: fs suite_branch: wip-vshankar-testing-20240125.044502 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 661946e7044723130c03627323f6a879a9848cb5 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0hyvbZJ9gvsAeRpvp7buH5ak1m4WEFb7CXwExj0AtpcaGc+4PY3ww6tbD9fsky7IpVAtVf4RU3ssYUskryAaQ= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3Pqla8kAnrFl5MUmD5SNHJkHK8VKHYPgL7gKNVcDPlOUMoIv55HIIobt+gKHf1QrNDEGole7gbhG/UFLXpUck= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI4tanWusQYW277aM8Ws84D77qIL+D1eqc0QTHN2sXmZS9Xyn2hBYpAfK9b3gnHtREHPbe9llcoIFdAineVZGFw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: c856e0f9244dd29a50697123b480d63f16a4ed81 timestamp: 2024-01-25_13:52:19 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-25T15:47:17.379 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa; will attempt to use it 2024-01-25T15:47:17.380 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks 2024-01-25T15:47:17.380 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-25T15:47:17.382 INFO:teuthology.task.internal:Checking packages... 2024-01-25T15:47:17.406 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '661946e7044723130c03627323f6a879a9848cb5' 2024-01-25T15:47:17.406 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-25T15:47:17.407 INFO:teuthology.packaging:ref: None 2024-01-25T15:47:17.407 INFO:teuthology.packaging:tag: None 2024-01-25T15:47:17.407 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240125.044502 2024-01-25T15:47:17.407 INFO:teuthology.packaging:sha1: 661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:47:17.407 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240125.044502 2024-01-25T15:47:17.638 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-885.g661946e7 2024-01-25T15:47:17.640 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-25T15:47:17.649 INFO:teuthology.task.internal:no buildpackages task found 2024-01-25T15:47:17.649 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-25T15:47:17.671 INFO:teuthology.task.internal:Saving configuration 2024-01-25T15:47:17.693 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-25T15:47:17.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-25T15:47:17.724 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.415543', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0hyvbZJ9gvsAeRpvp7buH5ak1m4WEFb7CXwExj0AtpcaGc+4PY3ww6tbD9fsky7IpVAtVf4RU3ssYUskryAaQ='} 2024-01-25T15:47:17.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.417553', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3Pqla8kAnrFl5MUmD5SNHJkHK8VKHYPgL7gKNVcDPlOUMoIv55HIIobt+gKHf1QrNDEGole7gbhG/UFLXpUck='} 2024-01-25T15:47:17.770 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.419315', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI4tanWusQYW277aM8Ws84D77qIL+D1eqc0QTHN2sXmZS9Xyn2hBYpAfK9b3gnHtREHPbe9llcoIFdAineVZGFw='} 2024-01-25T15:47:17.770 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-25T15:47:17.776 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-01-25T15:47:17.776 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-01-25T15:47:17.776 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-01-25T15:47:17.776 INFO:teuthology.run_tasks:Running task console_log... 2024-01-25T15:47:17.876 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fe8d9f9f160>, signals=[15]) 2024-01-25T15:47:17.877 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-25T15:47:17.883 INFO:teuthology.task.internal:Opening connections... 2024-01-25T15:47:17.883 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T15:47:17.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:47:17.995 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-01-25T15:47:17.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:47:18.089 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-01-25T15:47:18.090 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:47:18.175 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-25T15:47:18.182 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-01-25T15:47:18.201 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-01-25T15:47:18.201 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-01-25T15:47:18.259 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-01-25T15:47:18.259 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-01-25T15:47:18.259 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-01-25T15:47:18.259 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-01-25T15:47:18.259 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-25T15:47:18.260 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-25T15:47:18.261 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-01-25T15:47:18.284 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-01-25T15:47:18.302 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-01-25T15:47:18.302 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="CentOS Stream" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="9" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ID="centos" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="rhel fedora" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="9" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el9" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2024-01-25T15:47:18.359 INFO:teuthology.orchestra.run.smithi076.stdout:LOGO="fedora-logo-icon" 2024-01-25T15:47:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-25T15:47:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://centos.org/" 2024-01-25T15:47:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-25T15:47:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-25T15:47:18.360 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-25T15:47:18.360 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-01-25T15:47:18.384 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-01-25T15:47:18.402 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-01-25T15:47:18.402 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-25T15:47:18.459 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-25T15:47:18.460 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-25T15:47:18.460 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-01-25T15:47:18.484 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-25T15:47:18.495 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-25T15:47:18.501 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-25T15:47:18.501 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-01-25T15:47:18.504 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-01-25T15:47:18.506 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-01-25T15:47:18.523 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-25T15:47:18.529 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-25T15:47:18.529 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-01-25T15:47:18.561 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-01-25T15:47:18.563 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-01-25T15:47:18.617 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-25T15:47:18.697 INFO:teuthology.run_tasks:Running task kernel... 2024-01-25T15:47:18.711 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'} 2024-01-25T15:47:18.712 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-01-25T15:47:18.712 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-01-25T15:47:18.712 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-25T15:47:18.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:47:18.730 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-01-25T15:47:18.788 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-361.el9.x86_64 2024-01-25T15:47:18.788 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-361.el9.x86_64 2024-01-25T15:47:18.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-01-25T15:47:21.164 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 9.0 MB/s | 6.4 MB 00:00 2024-01-25T15:47:22.881 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 3.2 MB/s | 2.0 MB 00:00 2024-01-25T15:47:23.988 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-01-25T15:47:26.031 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-01-25T15:47:34.114 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 5.9 MB 00:04 2024-01-25T15:47:36.031 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 15 kB 00:00 2024-01-25T15:47:42.319 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 3.3 MB/s | 20 MB 00:06 2024-01-25T15:47:47.415 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-01-25T15:47:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-01-25T15:47:49.797 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-01-25T15:47:49.906 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-411.el9 CentOS-BaseOS 6.0 M 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 5.14.0-411.el9 CentOS-BaseOS 21 M 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 5.14.0-411.el9 CentOS-BaseOS 39 M 2024-01-25T15:47:49.920 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core x86_64 5.14.0-411.el9 CentOS-BaseOS 33 M 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 99 M 2024-01-25T15:47:49.921 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 122 M 2024-01-25T15:47:49.922 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-01-25T15:47:50.375 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): kernel-5.14.0-411.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-01-25T15:47:50.900 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): kernel-core-5.14.0-411.el9.x86_64.rpm 21 MB/s | 21 MB 00:00 2024-01-25T15:47:51.676 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): kernel-modules-core-5.14.0-411.el9.x86_6 26 MB/s | 33 MB 00:01 2024-01-25T15:47:52.092 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): kernel-modules-5.14.0-411.el9.x86_64.rpm 18 MB/s | 39 MB 00:02 2024-01-25T15:47:52.095 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-01-25T15:47:52.096 INFO:teuthology.orchestra.run.smithi033.stdout:Total 46 MB/s | 99 MB 00:02 2024-01-25T15:47:52.096 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-01-25T15:47:52.548 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-01-25T15:47:52.548 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-01-25T15:47:53.601 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-01-25T15:47:53.602 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-01-25T15:47:56.131 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-01-25T15:47:56.838 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-core-5.14.0-411.el9.x86_64 1/4 2024-01-25T15:47:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-25T15:47:59.936 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-25T15:48:00.120 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-25T15:48:05.180 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-25T15:48:05.291 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:10.378 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:41.153 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:41.154 INFO:teuthology.orchestra.run.smithi033.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-25T15:48:41.154 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:48:41.225 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:41.631 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:41.631 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-411.el9.x86_64 1/4 2024-01-25T15:48:41.631 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-5.14.0-411.el9.x86_64 2/4 2024-01-25T15:48:41.631 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-5.14.0-411.el9.x86_64 3/4 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-core-5.14.0-411.el9.x86_64 4/4 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:48:42.071 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-01-25T15:48:42.625 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-01-25T15:48:42.752 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.753 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-25T15:48:42.753 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-411.el9.x86_64 2024-01-25T15:48:42.754 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-01-25T15:48:42.754 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-01-25T15:48:42.754 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-01-25T15:48:42.754 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-01-25T15:48:42.754 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-01-25T15:48:42.867 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-411.el9.x86_64 2024-01-25T15:48:42.867 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-411.el9.x86_64 2024-01-25T15:48:42.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-01-25T15:48:42.912 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 68.el9 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8264254 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-01-25T15:48:42.913 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:48:42.914 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-01-25T15:48:42.915 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-25T15:48:42.915 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 5.14.0-411.el9.x86_64 2024-01-25T15:48:42.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-25T15:48:43.484 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-01-25T15:48:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-25T15:48:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-01-25T15:48:44.870 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-01-25T15:48:44.885 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.Q8K1Iho1bL 2024-01-25T15:48:44.886 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Q8K1Iho1bL 2024-01-25T15:48:44.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.Q8K1Iho1bL 2024-01-25T15:48:45.105 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.Q8K1Iho1bL is 6KB 2024-01-25T15:48:45.154 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.Q8K1Iho1bL 2024-01-25T15:48:45.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-01-25T15:48:45.238 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-25T15:48:45.238 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-25T15:48:45.238 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-25T15:48:45.238 INFO:teuthology.orchestra.run.smithi033.stdout:15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64.conf 2024-01-25T15:48:45.239 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-411.el9.x86_64 2024-01-25T15:48:45.399 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-01-25T15:49:15.402 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-01-25T15:49:15.402 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-01-25T15:49:15.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:50:15.405 DEBUG:teuthology.orchestra.remote:timed out 2024-01-25T15:50:24.407 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-01-25T15:50:24.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:50:24.842 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-01-25T15:50:25.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-01-25T15:50:25.257 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-411.el9.x86_64"... 2024-01-25T15:50:25.257 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-01-25T15:50:25.273 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-411.el9.x86_64 2024-01-25T15:50:25.273 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-411.el9.x86_64 vs 5.14.0-411.el9.x86_64 2024-01-25T15:50:25.273 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-25T15:50:25.273 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-25T15:50:26.274 DEBUG:teuthology.parallel:result is None 2024-01-25T15:50:26.274 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-25T15:50:26.282 INFO:teuthology.task.internal:Creating test directory... 2024-01-25T15:50:26.282 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-25T15:50:26.284 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-25T15:50:26.287 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-25T15:50:26.305 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-25T15:50:26.312 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-25T15:50:26.321 INFO:teuthology.task.internal:Creating archive directory... 2024-01-25T15:50:26.321 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-25T15:50:26.342 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-25T15:50:26.344 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-25T15:50:26.452 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-25T15:50:26.459 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-25T15:50:26.459 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-25T15:50:26.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:50:26.476 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-25T15:50:26.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:50:26.493 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-25T15:50:26.495 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-25T15:50:26.519 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-25T15:50:26.569 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.582 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.587 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.608 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.639 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-25T15:50:26.641 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-25T15:50:26.648 INFO:teuthology.task.internal:Configuring sudo... 2024-01-25T15:50:26.648 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-25T15:50:26.650 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-25T15:50:26.652 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-25T15:50:26.681 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-25T15:50:26.691 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-25T15:50:26.692 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-25T15:50:26.721 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-25T15:50:26.723 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-25T15:50:26.739 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:26.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:26.868 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:26.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:26.997 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:50:26.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-25T15:50:27.065 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:27.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:27.169 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:27.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:27.255 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:50:27.255 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-25T15:50:27.323 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:27.404 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-25T15:50:27.451 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:27.497 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-25T15:50:27.540 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:50:27.540 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-25T15:50:27.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-01-25T15:50:27.612 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-01-25T15:50:27.614 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-01-25T15:50:27.647 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T15:50:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T15:50:27.689 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T15:50:28.174 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-25T15:50:28.181 INFO:teuthology.task.internal:Starting timer... 2024-01-25T15:50:28.181 INFO:teuthology.run_tasks:Running task pcp... 2024-01-25T15:50:28.212 INFO:teuthology.run_tasks:Running task selinux... 2024-01-25T15:50:28.227 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-01-25T15:50:28.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-01-25T15:50:28.293 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-01-25T15:50:28.295 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2024-01-25T15:50:28.354 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2024-01-25T15:50:28.356 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-01-25T15:50:28.425 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-01-25T15:50:28.426 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-25T15:50:28.427 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-01-25T15:50:28.454 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-01-25T15:50:28.455 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2024-01-25T15:50:28.474 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2024-01-25T15:50:28.474 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-01-25T15:50:28.497 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-01-25T15:50:28.497 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2024-01-25T15:50:28.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T15:50:28.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:50:28.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T15:50:28.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:50:28.555 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T15:50:28.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T15:50:28.582 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-25T15:50:28.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-01-25T15:50:28.614 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2024-01-25T15:50:28.651 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-01-25T15:50:28.683 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-25T15:50:28.693 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-25T15:50:28.711 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-25T15:50:28.726 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi076.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-25T15:54:52.683 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-01-25T15:54:52.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-01-25T15:54:52.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:54:52.779 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-01-25T15:54:52.856 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-01-25T15:54:52.856 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-01-25T15:54:52.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:54:52.985 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-01-25T15:54:53.060 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-01-25T15:54:53.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-01-25T15:54:53.061 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-25T15:54:53.190 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-01-25T15:54:53.218 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-01-25T15:54:53.218 INFO:teuthology.run_tasks:Running task clock... 2024-01-25T15:54:53.228 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-25T15:54:53.229 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-25T15:54:53.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T15:54:53.231 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-25T15:54:53.232 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T15:54:53.234 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-25T15:54:53.234 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T15:54:53.263 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-25T15:54:53.265 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-25T15:54:53.281 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-25T15:54:53.283 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-25T15:54:53.291 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-25T15:54:53.308 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-25T15:54:53.311 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-01-25T15:54:53.313 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2024-01-25T15:54:53.326 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-01-25T15:54:53.327 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2024-01-25T15:54:53.338 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-01-25T15:54:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-25T15:54:53.343 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-25T15:54:53.352 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-01-25T15:54:53.359 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-25T15:54:53.360 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-25T15:54:53.369 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-25T15:54:53.385 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-25T15:54:53.404 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.407 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-01-25T15:54:53.414 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T15:54:53.415 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-01-25T15:54:53.415 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.415 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.415 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.415 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.428 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-01-25T15:54:53.431 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T15:54:53.431 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-01-25T15:54:53.432 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.432 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.432 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.432 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T15:54:53.433 INFO:teuthology.run_tasks:Running task install... 2024-01-25T15:54:53.441 DEBUG:teuthology.task.install:project ceph 2024-01-25T15:54:53.441 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'}} 2024-01-25T15:54:53.441 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '661946e7044723130c03627323f6a879a9848cb5'} 2024-01-25T15:54:53.441 INFO:teuthology.task.install:Using flavor: default 2024-01-25T15:54:53.447 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-01-25T15:54:53.447 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-25T15:54:53.448 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-25T15:54:53.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:54:53.450 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-25T15:54:53.450 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:54:53.451 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'tag': None, 'wait_for_package': False} 2024-01-25T15:54:53.452 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:54:53.598 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-25T15:54:53.598 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-25T15:54:53.690 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-25T15:54:53.690 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-25T15:54:53.716 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/ 2024-01-25T15:54:53.716 INFO:teuthology.task.install.rpm:Package version is 19.0.0-885.g661946e7 2024-01-25T15:54:53.854 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-25T15:54:53.854 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:54:53.854 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-25T15:54:53.883 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-25T15:54:53.883 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-25T15:54:53.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-25T15:54:53.955 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-25T15:54:53.955 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:54:53.955 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-25T15:54:53.984 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-25T15:54:53.984 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-25T15:54:54.024 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-25T15:54:54.024 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:54:54.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-25T15:54:54.032 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-25T15:54:54.053 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-25T15:54:54.053 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-25T15:54:54.054 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/661946e7044723130c03627323f6a879a9848cb5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-25T15:54:54.098 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-01-25T15:54:54.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-01-25T15:54:54.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-25T15:54:54.134 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-25T15:54:54.201 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-01-25T15:54:54.202 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-01-25T15:54:54.203 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-25T15:54:54.270 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2024-01-25T15:54:54.271 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2024-01-25T15:54:54.347 INFO:teuthology.orchestra.run.smithi033.stdout:74 files removed 2024-01-25T15:54:54.374 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-25T15:54:54.438 INFO:teuthology.orchestra.run.smithi152.stdout:74 files removed 2024-01-25T15:54:54.465 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-25T15:54:54.506 INFO:teuthology.orchestra.run.smithi076.stdout:74 files removed 2024-01-25T15:54:54.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-25T15:54:55.528 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 104 kB/s | 80 kB 00:00 2024-01-25T15:54:55.530 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 96 kB/s | 80 kB 00:00 2024-01-25T15:54:55.556 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 110 kB/s | 80 kB 00:00 2024-01-25T15:54:56.362 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2024-01-25T15:54:56.381 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2024-01-25T15:54:56.381 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 12 kB/s | 9.5 kB 00:00 2024-01-25T15:54:56.796 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 4.6 kB/s | 1.9 kB 00:00 2024-01-25T15:54:56.865 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 4.2 kB/s | 1.9 kB 00:00 2024-01-25T15:54:56.907 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 3.8 kB/s | 1.9 kB 00:00 2024-01-25T15:54:57.207 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 17 MB/s | 6.4 MB 00:00 2024-01-25T15:54:57.393 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2024-01-25T15:54:57.475 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2024-01-25T15:54:58.774 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-01-25T15:54:58.892 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS-9 - BaseOS 7.1 MB/s | 2.0 MB 00:00 2024-01-25T15:54:58.918 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2024-01-25T15:55:00.106 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-01-25T15:55:00.175 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 8.4 MB/s | 8.0 MB 00:00 2024-01-25T15:55:00.801 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-01-25T15:55:05.457 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 19 MB 00:04 2024-01-25T15:55:05.793 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - AppStream 4.8 MB/s | 19 MB 00:03 2024-01-25T15:55:06.992 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 19 MB 00:05 2024-01-25T15:55:09.869 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.9 MB 00:00 2024-01-25T15:55:10.908 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 5.9 MB 00:01 2024-01-25T15:55:11.167 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-01-25T15:55:11.369 INFO:teuthology.orchestra.run.smithi076.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-01-25T15:55:12.375 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 15 kB 00:00 2024-01-25T15:55:12.702 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-01-25T15:55:13.571 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 9.8 MB/s | 20 MB 00:02 2024-01-25T15:55:14.636 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-01-25T15:55:16.712 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 4.8 MB/s | 20 MB 00:04 2024-01-25T15:55:18.587 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 9.7 kB/s | 1.7 kB 00:00 2024-01-25T15:55:19.661 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-01-25T15:55:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:21.148 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:21.160 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-25T15:55:21.161 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-25T15:55:21.162 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-25T15:55:21.163 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:21.165 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:21.167 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-25T15:55:21.168 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-25T15:55:21.245 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================== 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================== 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-25T15:55:21.259 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-25T15:55:21.260 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-25T15:55:21.261 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-25T15:55:21.262 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-25T15:55:21.263 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-25T15:55:21.264 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-25T15:55:21.265 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-25T15:55:21.266 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-25T15:55:21.267 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-25T15:55:21.268 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-25T15:55:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-25T15:55:21.270 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================== 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:Install 139 Packages 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 16 Packages 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 209 M 2024-01-25T15:55:21.271 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2024-01-25T15:55:21.659 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-01-25T15:55:21.697 INFO:teuthology.orchestra.run.smithi076.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-01-25T15:55:22.379 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:22.380 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:22.395 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-25T15:55:22.396 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-25T15:55:22.397 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-25T15:55:22.398 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:22.400 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:22.403 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-25T15:55:22.404 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-25T15:55:22.486 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-01-25T15:55:22.507 INFO:teuthology.orchestra.run.smithi033.stdout:=========================================================================================== 2024-01-25T15:55:22.507 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-01-25T15:55:22.507 INFO:teuthology.orchestra.run.smithi033.stdout:=========================================================================================== 2024-01-25T15:55:22.507 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-25T15:55:22.508 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-25T15:55:22.509 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-25T15:55:22.510 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-25T15:55:22.511 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-25T15:55:22.512 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-25T15:55:22.513 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-25T15:55:22.514 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:22.515 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-25T15:55:22.516 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-25T15:55:22.517 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-25T15:55:22.518 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout:=========================================================================================== 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout:Install 139 Packages 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 16 Packages 2024-01-25T15:55:22.519 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:55:22.520 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 209 M 2024-01-25T15:55:22.520 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-01-25T15:55:23.055 INFO:teuthology.orchestra.run.smithi076.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 652 kB/s | 886 kB 00:01 2024-01-25T15:55:23.247 INFO:teuthology.orchestra.run.smithi076.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 759 kB/s | 145 kB 00:00 2024-01-25T15:55:23.403 INFO:teuthology.orchestra.run.smithi033.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2024-01-25T15:55:24.247 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:24.248 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-25T15:55:24.262 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-25T15:55:24.263 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-25T15:55:24.264 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-25T15:55:24.265 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:24.267 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-25T15:55:24.270 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-25T15:55:24.271 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-25T15:55:24.345 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================== 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================== 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.0.0-885.g661946e7.el9 ceph 6.5 k 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.0.0-885.g661946e7.el9 ceph 5.1 M 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 886 k 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-885.g661946e7.el9 ceph 145 k 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.0.0-885.g661946e7.el9 ceph 1.5 M 2024-01-25T15:55:24.360 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 133 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 1.7 M 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 7.4 M 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 51 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 8.7 M 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.0.0-885.g661946e7.el9 ceph 46 M 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 262 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 742 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 217 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.0.0-885.g661946e7.el9 ceph 27 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 18 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-25T15:55:24.361 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 31 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 743 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.0.0-885.g661946e7.el9 ceph 127 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.0.0-885.g661946e7.el9 ceph 162 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.0.0-885.g661946e7.el9 ceph 321 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 298 k 2024-01-25T15:55:24.362 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.0.0-885.g661946e7.el9 ceph 99 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.0.0-885.g661946e7.el9 ceph 85 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.0.0-885.g661946e7.el9 ceph 170 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-25T15:55:24.363 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.4 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 3.1 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 19 M 2024-01-25T15:55:24.364 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 28 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.0.0-885.g661946e7.el9 ceph 2.1 M 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 245 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.7 M 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.0.0-885.g661946e7.el9 ceph 17 M 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.0.0-885.g661946e7.el9 ceph-noarch 15 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.0.0-885.g661946e7.el9 ceph 25 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-25T15:55:24.365 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.0.0-885.g661946e7.el9 ceph 163 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.0.0-885.g661946e7.el9 ceph 499 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.0.0-885.g661946e7.el9 ceph 4.8 M 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-25T15:55:24.366 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.0.0-885.g661946e7.el9 ceph 45 k 2024-01-25T15:55:24.367 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.0.0-885.g661946e7.el9 ceph 130 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-25T15:55:24.368 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-25T15:55:24.369 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-25T15:55:24.370 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-25T15:55:24.371 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout:=========================================================================================== 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout:Install 139 Packages 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 16 Packages 2024-01-25T15:55:24.372 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:55:24.373 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 209 M 2024-01-25T15:55:24.373 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-01-25T15:55:24.604 INFO:teuthology.orchestra.run.smithi033.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 738 kB/s | 886 kB 00:01 2024-01-25T15:55:24.722 INFO:teuthology.orchestra.run.smithi076.stdout:(4/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 1.4 MB/s | 2.1 MB 00:01 2024-01-25T15:55:24.762 INFO:teuthology.orchestra.run.smithi033.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 919 kB/s | 145 kB 00:00 2024-01-25T15:55:25.222 INFO:teuthology.orchestra.run.smithi076.stdout:(5/155): ceph-base-19.0.0-885.g661946e7.el9.x86 1.4 MB/s | 5.1 MB 00:03 2024-01-25T15:55:25.414 INFO:teuthology.orchestra.run.smithi076.stdout:(6/155): ceph-common-19.0.0-885.g661946e7.el9.x 4.9 MB/s | 19 MB 00:03 2024-01-25T15:55:25.464 INFO:teuthology.orchestra.run.smithi076.stdout:(7/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 2.0 MB/s | 1.5 MB 00:00 2024-01-25T15:55:25.610 INFO:teuthology.orchestra.run.smithi152.stdout:(1/155): ceph-19.0.0-885.g661946e7.el9.x86_64.r 18 kB/s | 6.5 kB 00:00 2024-01-25T15:55:26.029 INFO:teuthology.orchestra.run.smithi033.stdout:(4/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 1.7 MB/s | 2.1 MB 00:01 2024-01-25T15:55:26.571 INFO:teuthology.orchestra.run.smithi033.stdout:(5/155): ceph-base-19.0.0-885.g661946e7.el9.x86 1.5 MB/s | 5.1 MB 00:03 2024-01-25T15:55:26.654 INFO:teuthology.orchestra.run.smithi033.stdout:(6/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 2.4 MB/s | 1.5 MB 00:00 2024-01-25T15:55:26.856 INFO:teuthology.orchestra.run.smithi076.stdout:(8/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 2.9 MB/s | 4.7 MB 00:01 2024-01-25T15:55:26.927 INFO:teuthology.orchestra.run.smithi152.stdout:(2/155): ceph-fuse-19.0.0-885.g661946e7.el9.x86 673 kB/s | 886 kB 00:01 2024-01-25T15:55:27.040 INFO:teuthology.orchestra.run.smithi076.stdout:(9/155): ceph-osd-19.0.0-885.g661946e7.el9.x86_ 10 MB/s | 17 MB 00:01 2024-01-25T15:55:27.065 INFO:teuthology.orchestra.run.smithi076.stdout:(10/155): ceph-selinux-19.0.0-885.g661946e7.el9 121 kB/s | 25 kB 00:00 2024-01-25T15:55:27.119 INFO:teuthology.orchestra.run.smithi152.stdout:(3/155): ceph-immutable-object-cache-19.0.0-885 758 kB/s | 145 kB 00:00 2024-01-25T15:55:27.215 INFO:teuthology.orchestra.run.smithi076.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 1.4 MB/s | 217 kB 00:00 2024-01-25T15:55:27.357 INFO:teuthology.orchestra.run.smithi076.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 191 kB/s | 27 kB 00:00 2024-01-25T15:55:27.507 INFO:teuthology.orchestra.run.smithi076.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 208 kB/s | 31 kB 00:00 2024-01-25T15:55:27.763 INFO:teuthology.orchestra.run.smithi033.stdout:(7/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 3.9 MB/s | 4.7 MB 00:01 2024-01-25T15:55:27.816 INFO:teuthology.orchestra.run.smithi076.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 2.4 MB/s | 743 kB 00:00 2024-01-25T15:55:27.958 INFO:teuthology.orchestra.run.smithi076.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 1.1 MB/s | 163 kB 00:00 2024-01-25T15:55:28.083 INFO:teuthology.orchestra.run.smithi076.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.0 MB/s | 127 kB 00:00 2024-01-25T15:55:28.308 INFO:teuthology.orchestra.run.smithi076.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 2.2 MB/s | 499 kB 00:00 2024-01-25T15:55:28.603 INFO:teuthology.orchestra.run.smithi152.stdout:(4/155): ceph-mds-19.0.0-885.g661946e7.el9.x86_ 1.4 MB/s | 2.1 MB 00:01 2024-01-25T15:55:29.203 INFO:teuthology.orchestra.run.smithi152.stdout:(5/155): ceph-mgr-19.0.0-885.g661946e7.el9.x86_ 2.5 MB/s | 1.5 MB 00:00 2024-01-25T15:55:29.508 INFO:teuthology.orchestra.run.smithi076.stdout:(18/155): ceph-radosgw-19.0.0-885.g661946e7.el9 2.1 MB/s | 8.7 MB 00:04 2024-01-25T15:55:29.650 INFO:teuthology.orchestra.run.smithi076.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 319 kB/s | 45 kB 00:00 2024-01-25T15:55:29.775 INFO:teuthology.orchestra.run.smithi076.stdout:(20/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 3.3 MB/s | 4.8 MB 00:01 2024-01-25T15:55:29.801 INFO:teuthology.orchestra.run.smithi076.stdout:(21/155): python3-ceph-common-19.0.0-885.g66194 864 kB/s | 130 kB 00:00 2024-01-25T15:55:29.901 INFO:teuthology.orchestra.run.smithi076.stdout:(22/155): python3-cephfs-19.0.0-885.g661946e7.e 1.3 MB/s | 162 kB 00:00 2024-01-25T15:55:30.026 INFO:teuthology.orchestra.run.smithi076.stdout:(23/155): python3-rados-19.0.0-885.g661946e7.el 1.4 MB/s | 321 kB 00:00 2024-01-25T15:55:30.060 INFO:teuthology.orchestra.run.smithi076.stdout:(24/155): python3-rbd-19.0.0-885.g661946e7.el9. 1.8 MB/s | 298 kB 00:00 2024-01-25T15:55:30.160 INFO:teuthology.orchestra.run.smithi076.stdout:(25/155): python3-rgw-19.0.0-885.g661946e7.el9. 745 kB/s | 99 kB 00:00 2024-01-25T15:55:30.193 INFO:teuthology.orchestra.run.smithi076.stdout:(26/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 640 kB/s | 85 kB 00:00 2024-01-25T15:55:30.280 INFO:teuthology.orchestra.run.smithi152.stdout:(6/155): ceph-base-19.0.0-885.g661946e7.el9.x86 1.0 MB/s | 5.1 MB 00:05 2024-01-25T15:55:30.327 INFO:teuthology.orchestra.run.smithi076.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 1.2 MB/s | 170 kB 00:00 2024-01-25T15:55:30.345 INFO:teuthology.orchestra.run.smithi152.stdout:(7/155): ceph-mon-19.0.0-885.g661946e7.el9.x86_ 4.1 MB/s | 4.7 MB 00:01 2024-01-25T15:55:30.453 INFO:teuthology.orchestra.run.smithi076.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 227 kB/s | 28 kB 00:00 2024-01-25T15:55:30.586 INFO:teuthology.orchestra.run.smithi076.stdout:(29/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 997 kB/s | 133 kB 00:00 2024-01-25T15:55:31.080 INFO:teuthology.orchestra.run.smithi033.stdout:(8/155): ceph-osd-19.0.0-885.g661946e7.el9.x86_ 3.8 MB/s | 17 MB 00:04 2024-01-25T15:55:31.196 INFO:teuthology.orchestra.run.smithi033.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 216 kB/s | 25 kB 00:00 2024-01-25T15:55:31.278 INFO:teuthology.orchestra.run.smithi076.stdout:(30/155): ceph-mgr-dashboard-19.0.0-885.g661946 2.5 MB/s | 1.7 MB 00:00 2024-01-25T15:55:31.421 INFO:teuthology.orchestra.run.smithi152.stdout:(8/155): ceph-radosgw-19.0.0-885.g661946e7.el9. 8.1 MB/s | 8.7 MB 00:01 2024-01-25T15:55:31.504 INFO:teuthology.orchestra.run.smithi152.stdout:(9/155): ceph-selinux-19.0.0-885.g661946e7.el9. 303 kB/s | 25 kB 00:00 2024-01-25T15:55:31.787 INFO:teuthology.orchestra.run.smithi076.stdout:(31/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 1.9 MB/s | 3.1 MB 00:01 2024-01-25T15:55:32.153 INFO:teuthology.orchestra.run.smithi076.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 669 kB/s | 245 kB 00:00 2024-01-25T15:55:32.287 INFO:teuthology.orchestra.run.smithi076.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 380 kB/s | 51 kB 00:00 2024-01-25T15:55:32.421 INFO:teuthology.orchestra.run.smithi076.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 113 kB/s | 15 kB 00:00 2024-01-25T15:55:32.768 INFO:teuthology.orchestra.run.smithi076.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 755 kB/s | 262 kB 00:00 2024-01-25T15:55:33.321 INFO:teuthology.orchestra.run.smithi076.stdout:(36/155): cephadm-19.0.0-885.g661946e7.el9.noar 1.3 MB/s | 742 kB 00:00 2024-01-25T15:55:33.438 INFO:teuthology.orchestra.run.smithi076.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 156 kB/s | 18 kB 00:00 2024-01-25T15:55:33.622 INFO:teuthology.orchestra.run.smithi076.stdout:(38/155): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-01-25T15:55:34.089 INFO:teuthology.orchestra.run.smithi076.stdout:(39/155): ceph-test-19.0.0-885.g661946e7.el9.x8 6.5 MB/s | 46 MB 00:07 2024-01-25T15:55:34.130 INFO:teuthology.orchestra.run.smithi076.stdout:(40/155): bison-3.7.4-5.el9.x86_64.rpm 1.8 MB/s | 920 kB 00:00 2024-01-25T15:55:34.156 INFO:teuthology.orchestra.run.smithi076.stdout:(41/155): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-01-25T15:55:34.181 INFO:teuthology.orchestra.run.smithi076.stdout:(42/155): elfutils-libelf-devel-0.190-2.el9.x86 410 kB/s | 20 kB 00:00 2024-01-25T15:55:34.206 INFO:teuthology.orchestra.run.smithi076.stdout:(43/155): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-01-25T15:55:34.231 INFO:teuthology.orchestra.run.smithi076.stdout:(44/155): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-01-25T15:55:34.340 INFO:teuthology.orchestra.run.smithi076.stdout:(45/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-01-25T15:55:34.365 INFO:teuthology.orchestra.run.smithi076.stdout:(46/155): flexiblas-openblas-openmp-3.0.4-8.el9 129 kB/s | 17 kB 00:00 2024-01-25T15:55:34.391 INFO:teuthology.orchestra.run.smithi076.stdout:(47/155): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-01-25T15:55:34.416 INFO:teuthology.orchestra.run.smithi076.stdout:(48/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-01-25T15:55:34.441 INFO:teuthology.orchestra.run.smithi076.stdout:(49/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 887 kB/s | 44 kB 00:00 2024-01-25T15:55:34.475 INFO:teuthology.orchestra.run.smithi076.stdout:(50/155): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 661 kB 00:00 2024-01-25T15:55:34.500 INFO:teuthology.orchestra.run.smithi076.stdout:(51/155): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-01-25T15:55:34.525 INFO:teuthology.orchestra.run.smithi076.stdout:(52/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 498 kB/s | 24 kB 00:00 2024-01-25T15:55:34.550 INFO:teuthology.orchestra.run.smithi076.stdout:(53/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2024-01-25T15:55:34.576 INFO:teuthology.orchestra.run.smithi076.stdout:(54/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-01-25T15:55:34.601 INFO:teuthology.orchestra.run.smithi076.stdout:(55/155): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-01-25T15:55:34.618 INFO:teuthology.orchestra.run.smithi076.stdout:(56/155): openblas-0.3.21-2.el9.x86_64.rpm 825 kB/s | 34 kB 00:00 2024-01-25T15:55:34.621 INFO:teuthology.orchestra.run.smithi033.stdout:(10/155): ceph-radosgw-19.0.0-885.g661946e7.el9 1.3 MB/s | 8.7 MB 00:06 2024-01-25T15:55:34.710 INFO:teuthology.orchestra.run.smithi076.stdout:(57/155): ceph-mgr-diskprediction-local-19.0.0- 2.2 MB/s | 7.4 MB 00:03 2024-01-25T15:55:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 2.3 MB/s | 217 kB 00:00 2024-01-25T15:55:34.746 INFO:teuthology.orchestra.run.smithi033.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 816 kB/s | 27 kB 00:00 2024-01-25T15:55:34.780 INFO:teuthology.orchestra.run.smithi033.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 940 kB/s | 31 kB 00:00 2024-01-25T15:55:34.787 INFO:teuthology.orchestra.run.smithi152.stdout:(10/155): ceph-test-19.0.0-885.g661946e7.el9.x8 14 MB/s | 46 MB 00:03 2024-01-25T15:55:34.802 INFO:teuthology.orchestra.run.smithi076.stdout:(58/155): openblas-openmp-0.3.21-2.el9.x86_64.r 24 MB/s | 4.7 MB 00:00 2024-01-25T15:55:34.837 INFO:teuthology.orchestra.run.smithi152.stdout:(11/155): cephfs-mirror-19.0.0-885.g661946e7.el 4.3 MB/s | 217 kB 00:00 2024-01-25T15:55:34.888 INFO:teuthology.orchestra.run.smithi152.stdout:(12/155): cephfs-shell-19.0.0-885.g661946e7.el9 544 kB/s | 27 kB 00:00 2024-01-25T15:55:34.902 INFO:teuthology.orchestra.run.smithi076.stdout:(59/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-01-25T15:55:34.930 INFO:teuthology.orchestra.run.smithi152.stdout:(13/155): libcephfs-devel-19.0.0-885.g661946e7. 748 kB/s | 31 kB 00:00 2024-01-25T15:55:34.944 INFO:teuthology.orchestra.run.smithi076.stdout:(60/155): postgresql-13.11-1.el9.x86_64.rpm 6.6 MB/s | 1.5 MB 00:00 2024-01-25T15:55:34.978 INFO:teuthology.orchestra.run.smithi076.stdout:(61/155): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2024-01-25T15:55:34.996 INFO:teuthology.orchestra.run.smithi152.stdout:(14/155): libcephfs2-19.0.0-885.g661946e7.el9.x 11 MB/s | 743 kB 00:00 2024-01-25T15:55:35.003 INFO:teuthology.orchestra.run.smithi076.stdout:(62/155): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-01-25T15:55:35.047 INFO:teuthology.orchestra.run.smithi152.stdout:(15/155): libcephsqlite-19.0.0-885.g661946e7.el 3.2 MB/s | 163 kB 00:00 2024-01-25T15:55:35.055 INFO:teuthology.orchestra.run.smithi033.stdout:(14/155): ceph-common-19.0.0-885.g661946e7.el9. 1.6 MB/s | 19 MB 00:11 2024-01-25T15:55:35.097 INFO:teuthology.orchestra.run.smithi152.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 2.5 MB/s | 127 kB 00:00 2024-01-25T15:55:35.114 INFO:teuthology.orchestra.run.smithi033.stdout:(15/155): libcephfs2-19.0.0-885.g661946e7.el9.x 2.2 MB/s | 743 kB 00:00 2024-01-25T15:55:35.164 INFO:teuthology.orchestra.run.smithi152.stdout:(17/155): libradosstriper1-19.0.0-885.g661946e7 7.3 MB/s | 499 kB 00:00 2024-01-25T15:55:35.194 INFO:teuthology.orchestra.run.smithi076.stdout:(63/155): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2024-01-25T15:55:35.197 INFO:teuthology.orchestra.run.smithi033.stdout:(16/155): librados-devel-19.0.0-885.g661946e7.e 1.5 MB/s | 127 kB 00:00 2024-01-25T15:55:35.220 INFO:teuthology.orchestra.run.smithi076.stdout:(64/155): python3-attrs-20.3.0-7.el9.noarch.rpm 320 kB/s | 78 kB 00:00 2024-01-25T15:55:35.256 INFO:teuthology.orchestra.run.smithi033.stdout:(17/155): libcephsqlite-19.0.0-885.g661946e7.el 814 kB/s | 163 kB 00:00 2024-01-25T15:55:35.345 INFO:teuthology.orchestra.run.smithi076.stdout:(65/155): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 5.8 MB 00:00 2024-01-25T15:55:35.371 INFO:teuthology.orchestra.run.smithi076.stdout:(66/155): python3-jinja2-2.11.3-4.el9.noarch.rp 1.3 MB/s | 228 kB 00:00 2024-01-25T15:55:35.396 INFO:teuthology.orchestra.run.smithi076.stdout:(67/155): python3-jsonpatch-1.21-16.el9.noarch. 141 kB/s | 24 kB 00:00 2024-01-25T15:55:35.421 INFO:teuthology.orchestra.run.smithi076.stdout:(68/155): python3-jsonpointer-2.0-4.el9.noarch. 241 kB/s | 18 kB 00:00 2024-01-25T15:55:35.446 INFO:teuthology.orchestra.run.smithi076.stdout:(69/155): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-01-25T15:55:35.471 INFO:teuthology.orchestra.run.smithi076.stdout:(70/155): python3-markupsafe-1.1.1-12.el9.x86_6 420 kB/s | 32 kB 00:00 2024-01-25T15:55:35.473 INFO:teuthology.orchestra.run.smithi152.stdout:(18/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 16 MB/s | 4.8 MB 00:00 2024-01-25T15:55:35.490 INFO:teuthology.orchestra.run.smithi033.stdout:(18/155): libradosstriper1-19.0.0-885.g661946e7 1.7 MB/s | 499 kB 00:00 2024-01-25T15:55:35.515 INFO:teuthology.orchestra.run.smithi152.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 1.1 MB/s | 45 kB 00:00 2024-01-25T15:55:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:(19/155): python3-ceph-argparse-19.0.0-885.g661 1.1 MB/s | 45 kB 00:00 2024-01-25T15:55:35.565 INFO:teuthology.orchestra.run.smithi152.stdout:(20/155): python3-ceph-common-19.0.0-885.g66194 2.5 MB/s | 130 kB 00:00 2024-01-25T15:55:35.599 INFO:teuthology.orchestra.run.smithi033.stdout:(20/155): python3-ceph-common-19.0.0-885.g66194 1.9 MB/s | 130 kB 00:00 2024-01-25T15:55:35.605 INFO:teuthology.orchestra.run.smithi076.stdout:(71/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-01-25T15:55:35.615 INFO:teuthology.orchestra.run.smithi152.stdout:(21/155): python3-cephfs-19.0.0-885.g661946e7.e 3.2 MB/s | 162 kB 00:00 2024-01-25T15:55:35.630 INFO:teuthology.orchestra.run.smithi076.stdout:(72/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 217 kB 00:00 2024-01-25T15:55:35.655 INFO:teuthology.orchestra.run.smithi076.stdout:(73/155): python3-oauthlib-3.1.1-5.el9.noarch.r 963 kB/s | 174 kB 00:00 2024-01-25T15:55:35.674 INFO:teuthology.orchestra.run.smithi152.stdout:(22/155): python3-rados-19.0.0-885.g661946e7.el 5.4 MB/s | 321 kB 00:00 2024-01-25T15:55:35.674 INFO:teuthology.orchestra.run.smithi033.stdout:(21/155): python3-cephfs-19.0.0-885.g661946e7.e 2.1 MB/s | 162 kB 00:00 2024-01-25T15:55:35.681 INFO:teuthology.orchestra.run.smithi076.stdout:(74/155): python3-prettytable-0.7.2-27.el9.noar 548 kB/s | 41 kB 00:00 2024-01-25T15:55:35.706 INFO:teuthology.orchestra.run.smithi076.stdout:(75/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-01-25T15:55:35.731 INFO:teuthology.orchestra.run.smithi076.stdout:(76/155): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-01-25T15:55:35.733 INFO:teuthology.orchestra.run.smithi152.stdout:(23/155): python3-rbd-19.0.0-885.g661946e7.el9. 5.0 MB/s | 298 kB 00:00 2024-01-25T15:55:35.756 INFO:teuthology.orchestra.run.smithi076.stdout:(77/155): python3-pytz-2021.1-5.el9.noarch.rpm 629 kB/s | 47 kB 00:00 2024-01-25T15:55:35.775 INFO:teuthology.orchestra.run.smithi152.stdout:(24/155): python3-rgw-19.0.0-885.g661946e7.el9. 2.3 MB/s | 99 kB 00:00 2024-01-25T15:55:35.781 INFO:teuthology.orchestra.run.smithi076.stdout:(78/155): python3-requests-oauthlib-1.3.0-12.el 856 kB/s | 43 kB 00:00 2024-01-25T15:55:35.799 INFO:teuthology.orchestra.run.smithi033.stdout:(22/155): python3-rados-19.0.0-885.g661946e7.el 2.5 MB/s | 321 kB 00:00 2024-01-25T15:55:35.825 INFO:teuthology.orchestra.run.smithi152.stdout:(25/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-01-25T15:55:35.925 INFO:teuthology.orchestra.run.smithi033.stdout:(23/155): python3-rbd-19.0.0-885.g661946e7.el9. 2.3 MB/s | 298 kB 00:00 2024-01-25T15:55:36.000 INFO:teuthology.orchestra.run.smithi076.stdout:(79/155): python3-toml-0.10.2-6.el9.noarch.rpm 160 kB/s | 39 kB 00:00 2024-01-25T15:55:36.008 INFO:teuthology.orchestra.run.smithi033.stdout:(24/155): python3-rgw-19.0.0-885.g661946e7.el9. 1.2 MB/s | 99 kB 00:00 2024-01-25T15:55:36.024 INFO:teuthology.orchestra.run.smithi076.stdout:(80/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 170 kB/s | 41 kB 00:00 2024-01-25T15:55:36.042 INFO:teuthology.orchestra.run.smithi152.stdout:(26/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 14 MB/s | 3.1 MB 00:00 2024-01-25T15:55:36.050 INFO:teuthology.orchestra.run.smithi033.stdout:(25/155): rbd-fuse-19.0.0-885.g661946e7.el9.x86 2.0 MB/s | 85 kB 00:00 2024-01-25T15:55:36.084 INFO:teuthology.orchestra.run.smithi152.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 4.0 MB/s | 170 kB 00:00 2024-01-25T15:55:36.126 INFO:teuthology.orchestra.run.smithi152.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 677 kB/s | 28 kB 00:00 2024-01-25T15:55:36.168 INFO:teuthology.orchestra.run.smithi152.stdout:(29/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 3.1 MB/s | 133 kB 00:00 2024-01-25T15:55:36.217 INFO:teuthology.orchestra.run.smithi076.stdout:(81/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-01-25T15:55:36.241 INFO:teuthology.orchestra.run.smithi076.stdout:(82/155): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 300 kB 00:00 2024-01-25T15:55:36.266 INFO:teuthology.orchestra.run.smithi076.stdout:(83/155): uuid-1.6.2-55.el9.x86_64.rpm 234 kB/s | 56 kB 00:00 2024-01-25T15:55:36.283 INFO:teuthology.orchestra.run.smithi076.stdout:(84/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2024-01-25T15:55:36.308 INFO:teuthology.orchestra.run.smithi076.stdout:(85/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-01-25T15:55:36.310 INFO:teuthology.orchestra.run.smithi152.stdout:(30/155): ceph-mgr-dashboard-19.0.0-885.g661946 12 MB/s | 1.7 MB 00:00 2024-01-25T15:55:36.333 INFO:teuthology.orchestra.run.smithi076.stdout:(86/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 660 kB/s | 44 kB 00:00 2024-01-25T15:55:36.358 INFO:teuthology.orchestra.run.smithi076.stdout:(87/155): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-01-25T15:55:36.392 INFO:teuthology.orchestra.run.smithi076.stdout:(88/155): jq-1.6-16.el9.x86_64.rpm 3.1 MB/s | 186 kB 00:00 2024-01-25T15:55:36.425 INFO:teuthology.orchestra.run.smithi076.stdout:(89/155): libgfortran-11.4.1-3.el9.x86_64.rpm 8.5 MB/s | 802 kB 00:00 2024-01-25T15:55:36.459 INFO:teuthology.orchestra.run.smithi076.stdout:(90/155): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 194 kB 00:00 2024-01-25T15:55:36.484 INFO:teuthology.orchestra.run.smithi076.stdout:(91/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-01-25T15:55:36.510 INFO:teuthology.orchestra.run.smithi076.stdout:(92/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-01-25T15:55:36.543 INFO:teuthology.orchestra.run.smithi076.stdout:(93/155): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-01-25T15:55:36.568 INFO:teuthology.orchestra.run.smithi076.stdout:(94/155): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-25T15:55:36.603 INFO:teuthology.orchestra.run.smithi076.stdout:(95/155): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-01-25T15:55:36.627 INFO:teuthology.orchestra.run.smithi076.stdout:(96/155): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2024-01-25T15:55:36.670 INFO:teuthology.orchestra.run.smithi076.stdout:(97/155): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 187 kB 00:00 2024-01-25T15:55:36.761 INFO:teuthology.orchestra.run.smithi076.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-01-25T15:55:36.777 INFO:teuthology.orchestra.run.smithi152.stdout:(31/155): ceph-mgr-diskprediction-local-19.0.0- 16 MB/s | 7.4 MB 00:00 2024-01-25T15:55:36.836 INFO:teuthology.orchestra.run.smithi152.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 4.1 MB/s | 245 kB 00:00 2024-01-25T15:55:36.878 INFO:teuthology.orchestra.run.smithi152.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 1.2 MB/s | 51 kB 00:00 2024-01-25T15:55:36.911 INFO:teuthology.orchestra.run.smithi152.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 456 kB/s | 15 kB 00:00 2024-01-25T15:55:36.936 INFO:teuthology.orchestra.run.smithi076.stdout:(99/155): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.1 MB/s | 309 kB 00:00 2024-01-25T15:55:36.962 INFO:teuthology.orchestra.run.smithi152.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 5.1 MB/s | 262 kB 00:00 2024-01-25T15:55:37.009 INFO:teuthology.orchestra.run.smithi033.stdout:(26/155): rbd-mirror-19.0.0-885.g661946e7.el9.x 3.2 MB/s | 3.1 MB 00:00 2024-01-25T15:55:37.020 INFO:teuthology.orchestra.run.smithi076.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 314 kB/s | 26 kB 00:00 2024-01-25T15:55:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:(27/155): rbd-nbd-19.0.0-885.g661946e7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-01-25T15:55:37.087 INFO:teuthology.orchestra.run.smithi076.stdout:(101/155): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-01-25T15:55:37.112 INFO:teuthology.orchestra.run.smithi076.stdout:(102/155): liboath-2.6.7-2.el9.x86_64.rpm 539 kB/s | 49 kB 00:00 2024-01-25T15:55:37.117 INFO:teuthology.orchestra.run.smithi033.stdout:(28/155): ceph-grafana-dashboards-19.0.0-885.g6 681 kB/s | 28 kB 00:00 2024-01-25T15:55:37.137 INFO:teuthology.orchestra.run.smithi076.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-01-25T15:55:37.145 INFO:teuthology.orchestra.run.smithi152.stdout:(36/155): ceph-osd-19.0.0-885.g661946e7.el9.x86 2.4 MB/s | 17 MB 00:06 2024-01-25T15:55:37.180 INFO:teuthology.orchestra.run.smithi152.stdout:(37/155): cephadm-19.0.0-885.g661946e7.el9.noar 3.3 MB/s | 742 kB 00:00 2024-01-25T15:55:37.184 INFO:teuthology.orchestra.run.smithi033.stdout:(29/155): ceph-mgr-cephadm-19.0.0-885.g661946e7 1.9 MB/s | 133 kB 00:00 2024-01-25T15:55:37.204 INFO:teuthology.orchestra.run.smithi152.stdout:(38/155): cephfs-top-19.0.0-885.g661946e7.el9.n 312 kB/s | 18 kB 00:00 2024-01-25T15:55:37.280 INFO:teuthology.orchestra.run.smithi076.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 907 kB/s | 151 kB 00:00 2024-01-25T15:55:37.313 INFO:teuthology.orchestra.run.smithi076.stdout:(105/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-01-25T15:55:37.402 INFO:teuthology.orchestra.run.smithi033.stdout:(30/155): librgw2-19.0.0-885.g661946e7.el9.x86_ 2.2 MB/s | 4.8 MB 00:02 2024-01-25T15:55:37.405 INFO:teuthology.orchestra.run.smithi076.stdout:(106/155): python3-devel-3.9.17-2.el9.x86_64.rp 306 kB/s | 245 kB 00:00 2024-01-25T15:55:37.439 INFO:teuthology.orchestra.run.smithi076.stdout:(107/155): python3-asyncssh-2.13.2-3.el9.noarch 3.4 MB/s | 548 kB 00:00 2024-01-25T15:55:37.464 INFO:teuthology.orchestra.run.smithi076.stdout:(108/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 289 kB/s | 43 kB 00:00 2024-01-25T15:55:37.489 INFO:teuthology.orchestra.run.smithi076.stdout:(109/155): python3-cachetools-4.2.4-1.el9.noarc 387 kB/s | 32 kB 00:00 2024-01-25T15:55:37.514 INFO:teuthology.orchestra.run.smithi076.stdout:(110/155): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-01-25T15:55:37.540 INFO:teuthology.orchestra.run.smithi076.stdout:(111/155): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-01-25T15:55:37.545 INFO:teuthology.orchestra.run.smithi033.stdout:(31/155): ceph-mgr-dashboard-19.0.0-885.g661946 4.9 MB/s | 1.7 MB 00:00 2024-01-25T15:55:37.573 INFO:teuthology.orchestra.run.smithi076.stdout:(112/155): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-01-25T15:55:37.584 INFO:teuthology.orchestra.run.smithi152.stdout:(39/155): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 662 kB 00:00 2024-01-25T15:55:37.611 INFO:teuthology.orchestra.run.smithi033.stdout:(32/155): ceph-mgr-modules-core-19.0.0-885.g661 3.6 MB/s | 245 kB 00:00 2024-01-25T15:55:37.640 INFO:teuthology.orchestra.run.smithi076.stdout:(113/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-01-25T15:55:37.642 INFO:teuthology.orchestra.run.smithi152.stdout:(40/155): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-01-25T15:55:37.644 INFO:teuthology.orchestra.run.smithi033.stdout:(33/155): ceph-mgr-rook-19.0.0-885.g661946e7.el 1.5 MB/s | 51 kB 00:00 2024-01-25T15:55:37.665 INFO:teuthology.orchestra.run.smithi076.stdout:(114/155): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-01-25T15:55:37.680 INFO:teuthology.orchestra.run.smithi033.stdout:(34/155): ceph-prometheus-alerts-19.0.0-885.g66 457 kB/s | 15 kB 00:00 2024-01-25T15:55:37.705 INFO:teuthology.orchestra.run.smithi152.stdout:(41/155): elfutils-libelf-devel-0.190-2.el9.x86 306 kB/s | 20 kB 00:00 2024-01-25T15:55:37.741 INFO:teuthology.orchestra.run.smithi152.stdout:(42/155): bison-3.7.4-5.el9.x86_64.rpm 1.7 MB/s | 920 kB 00:00 2024-01-25T15:55:37.741 INFO:teuthology.orchestra.run.smithi076.stdout:(115/155): python3-google-auth-2.26.2-1.el9.noa 1.2 MB/s | 202 kB 00:00 2024-01-25T15:55:37.744 INFO:teuthology.orchestra.run.smithi033.stdout:(35/155): ceph-volume-19.0.0-885.g661946e7.el9. 3.8 MB/s | 262 kB 00:00 2024-01-25T15:55:37.766 INFO:teuthology.orchestra.run.smithi076.stdout:(116/155): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-01-25T15:55:37.781 INFO:teuthology.orchestra.run.smithi152.stdout:(43/155): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-01-25T15:55:37.791 INFO:teuthology.orchestra.run.smithi076.stdout:(117/155): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-01-25T15:55:37.809 INFO:teuthology.orchestra.run.smithi152.stdout:(44/155): flexiblas-3.0.4-8.el9.x86_64.rpm 473 kB/s | 32 kB 00:00 2024-01-25T15:55:37.883 INFO:teuthology.orchestra.run.smithi076.stdout:(118/155): python3-jaraco-collections-3.0.0-8.e 163 kB/s | 23 kB 00:00 2024-01-25T15:55:37.896 INFO:teuthology.orchestra.run.smithi033.stdout:(36/155): cephadm-19.0.0-885.g661946e7.el9.noar 4.8 MB/s | 742 kB 00:00 2024-01-25T15:55:37.908 INFO:teuthology.orchestra.run.smithi076.stdout:(119/155): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-01-25T15:55:37.928 INFO:teuthology.orchestra.run.smithi033.stdout:(37/155): cephfs-top-19.0.0-885.g661946e7.el9.n 548 kB/s | 18 kB 00:00 2024-01-25T15:55:37.933 INFO:teuthology.orchestra.run.smithi076.stdout:(120/155): python3-jaraco-text-3.2.0-6.el9.noar 138 kB/s | 20 kB 00:00 2024-01-25T15:55:37.974 INFO:teuthology.orchestra.run.smithi152.stdout:(45/155): flexiblas-openblas-openmp-3.0.4-8.el9 103 kB/s | 17 kB 00:00 2024-01-25T15:55:37.992 INFO:teuthology.orchestra.run.smithi076.stdout:(121/155): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-01-25T15:55:38.025 INFO:teuthology.orchestra.run.smithi076.stdout:(122/155): python3-logutils-0.3.5-21.el9.noarch 505 kB/s | 46 kB 00:00 2024-01-25T15:55:38.050 INFO:teuthology.orchestra.run.smithi076.stdout:(123/155): python3-more-itertools-8.12.0-2.el9. 674 kB/s | 79 kB 00:00 2024-01-25T15:55:38.076 INFO:teuthology.orchestra.run.smithi076.stdout:(124/155): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-01-25T15:55:38.101 INFO:teuthology.orchestra.run.smithi076.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-01-25T15:55:38.126 INFO:teuthology.orchestra.run.smithi076.stdout:(126/155): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-01-25T15:55:38.149 INFO:teuthology.orchestra.run.smithi152.stdout:(46/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.1 MB/s | 3.0 MB 00:00 2024-01-25T15:55:38.152 INFO:teuthology.orchestra.run.smithi076.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-01-25T15:55:38.173 INFO:teuthology.orchestra.run.smithi152.stdout:(47/155): libaio-devel-0.3.111-13.el9.x86_64.rp 50 kB/s | 10 kB 00:00 2024-01-25T15:55:38.177 INFO:teuthology.orchestra.run.smithi076.stdout:(128/155): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-01-25T15:55:38.219 INFO:teuthology.orchestra.run.smithi076.stdout:(129/155): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-01-25T15:55:38.232 INFO:teuthology.orchestra.run.smithi152.stdout:(48/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 760 kB/s | 44 kB 00:00 2024-01-25T15:55:38.244 INFO:teuthology.orchestra.run.smithi076.stdout:(130/155): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-01-25T15:55:38.257 INFO:teuthology.orchestra.run.smithi152.stdout:(49/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-01-25T15:55:38.269 INFO:teuthology.orchestra.run.smithi076.stdout:(131/155): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-01-25T15:55:38.278 INFO:teuthology.orchestra.run.smithi033.stdout:(38/155): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2024-01-25T15:55:38.294 INFO:teuthology.orchestra.run.smithi076.stdout:(132/155): python3-tempora-5.0.0-2.el9.noarch.r 478 kB/s | 36 kB 00:00 2024-01-25T15:55:38.316 INFO:teuthology.orchestra.run.smithi152.stdout:(50/155): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-01-25T15:55:38.319 INFO:teuthology.orchestra.run.smithi076.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-01-25T15:55:38.345 INFO:teuthology.orchestra.run.smithi076.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-01-25T15:55:38.350 INFO:teuthology.orchestra.run.smithi152.stdout:(51/155): libtool-2.4.6-45.el9.x86_64.rpm 6.1 MB/s | 578 kB 00:00 2024-01-25T15:55:38.362 INFO:teuthology.orchestra.run.smithi033.stdout:(39/155): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 920 kB 00:00 2024-01-25T15:55:38.370 INFO:teuthology.orchestra.run.smithi076.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-01-25T15:55:38.375 INFO:teuthology.orchestra.run.smithi152.stdout:(52/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 425 kB/s | 24 kB 00:00 2024-01-25T15:55:38.396 INFO:teuthology.orchestra.run.smithi076.stdout:(136/155): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-01-25T15:55:38.396 INFO:teuthology.orchestra.run.smithi033.stdout:(40/155): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-01-25T15:55:38.400 INFO:teuthology.orchestra.run.smithi152.stdout:(53/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 925 kB/s | 46 kB 00:00 2024-01-25T15:55:38.421 INFO:teuthology.orchestra.run.smithi076.stdout:(137/155): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-01-25T15:55:38.421 INFO:teuthology.orchestra.run.smithi033.stdout:(41/155): elfutils-libelf-devel-0.190-2.el9.x86 826 kB/s | 20 kB 00:00 2024-01-25T15:55:38.434 INFO:teuthology.orchestra.run.smithi152.stdout:(54/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.7 MB/s | 282 kB 00:00 2024-01-25T15:55:38.460 INFO:teuthology.orchestra.run.smithi152.stdout:(55/155): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 187 kB 00:00 2024-01-25T15:55:38.463 INFO:teuthology.orchestra.run.smithi033.stdout:(42/155): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-01-25T15:55:38.485 INFO:teuthology.orchestra.run.smithi152.stdout:(56/155): openblas-0.3.21-2.el9.x86_64.rpm 680 kB/s | 34 kB 00:00 2024-01-25T15:55:38.488 INFO:teuthology.orchestra.run.smithi033.stdout:(43/155): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2024-01-25T15:55:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:(138/155): re2-20211101-3.el9.x86_64.rpm 1.5 MB/s | 192 kB 00:00 2024-01-25T15:55:38.546 INFO:teuthology.orchestra.run.smithi076.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-01-25T15:55:38.597 INFO:teuthology.orchestra.run.smithi076.stdout:(140/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2024-01-25T15:55:38.622 INFO:teuthology.orchestra.run.smithi033.stdout:(44/155): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-01-25T15:55:38.656 INFO:teuthology.orchestra.run.smithi033.stdout:(45/155): flexiblas-openblas-openmp-3.0.4-8.el9 514 kB/s | 17 kB 00:00 2024-01-25T15:55:38.705 INFO:teuthology.orchestra.run.smithi076.stdout:(141/155): NetworkManager-1.45.10-1.el9.x86_64. 21 MB/s | 2.2 MB 00:00 2024-01-25T15:55:38.706 INFO:teuthology.orchestra.run.smithi033.stdout:(46/155): libaio-devel-0.3.111-13.el9.x86_64.rp 202 kB/s | 10 kB 00:00 2024-01-25T15:55:38.748 INFO:teuthology.orchestra.run.smithi033.stdout:(47/155): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 159 kB 00:00 2024-01-25T15:55:38.798 INFO:teuthology.orchestra.run.smithi033.stdout:(48/155): librabbitmq-0.11.0-7.el9.x86_64.rpm 884 kB/s | 44 kB 00:00 2024-01-25T15:55:38.814 INFO:teuthology.orchestra.run.smithi076.stdout:(142/155): NetworkManager-libnm-1.45.10-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-01-25T15:55:38.827 INFO:teuthology.orchestra.run.smithi152.stdout:(57/155): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-01-25T15:55:38.856 INFO:teuthology.orchestra.run.smithi076.stdout:(143/155): NetworkManager-team-1.45.10-1.el9.x8 1.0 MB/s | 42 kB 00:00 2024-01-25T15:55:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:(49/155): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-01-25T15:55:38.906 INFO:teuthology.orchestra.run.smithi076.stdout:(144/155): NetworkManager-tui-1.45.10-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-01-25T15:55:38.915 INFO:teuthology.orchestra.run.smithi033.stdout:(50/155): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-01-25T15:55:38.956 INFO:teuthology.orchestra.run.smithi076.stdout:(145/155): acl-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 69 kB 00:00 2024-01-25T15:55:38.960 INFO:teuthology.orchestra.run.smithi152.stdout:(58/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 6.3 MB/s | 3.0 MB 00:00 2024-01-25T15:55:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:(51/155): libuuid-devel-2.37.4-15.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-01-25T15:55:38.998 INFO:teuthology.orchestra.run.smithi076.stdout:(146/155): elfutils-debuginfod-client-0.190-2.e 849 kB/s | 35 kB 00:00 2024-01-25T15:55:39.002 INFO:teuthology.orchestra.run.smithi152.stdout:(59/155): postgresql-13.11-1.el9.x86_64.rpm 8.8 MB/s | 1.5 MB 00:00 2024-01-25T15:55:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:(52/155): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 46 kB 00:00 2024-01-25T15:55:39.044 INFO:teuthology.orchestra.run.smithi152.stdout:(60/155): postgresql-contrib-13.11-1.el9.x86_64 9.8 MB/s | 811 kB 00:00 2024-01-25T15:55:39.048 INFO:teuthology.orchestra.run.smithi076.stdout:(147/155): elfutils-libelf-0.190-2.el9.x86_64.r 3.8 MB/s | 191 kB 00:00 2024-01-25T15:55:39.058 INFO:teuthology.orchestra.run.smithi033.stdout:(53/155): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-01-25T15:55:39.070 INFO:teuthology.orchestra.run.smithi152.stdout:(61/155): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-01-25T15:55:39.100 INFO:teuthology.orchestra.run.smithi033.stdout:(54/155): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-01-25T15:55:39.107 INFO:teuthology.orchestra.run.smithi076.stdout:(148/155): elfutils-libs-0.190-2.el9.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2024-01-25T15:55:39.142 INFO:teuthology.orchestra.run.smithi033.stdout:(55/155): openblas-0.3.21-2.el9.x86_64.rpm 818 kB/s | 34 kB 00:00 2024-01-25T15:55:39.149 INFO:teuthology.orchestra.run.smithi076.stdout:(149/155): libacl-2.3.1-4.el9.x86_64.rpm 527 kB/s | 22 kB 00:00 2024-01-25T15:55:39.257 INFO:teuthology.orchestra.run.smithi076.stdout:(150/155): openssl-3.0.7-25.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-01-25T15:55:39.359 INFO:teuthology.orchestra.run.smithi033.stdout:(56/155): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-01-25T15:55:39.383 INFO:teuthology.orchestra.run.smithi076.stdout:(151/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-01-25T15:55:39.445 INFO:teuthology.orchestra.run.smithi152.stdout:(62/155): ceph-common-19.0.0-885.g661946e7.el9. 1.4 MB/s | 19 MB 00:14 2024-01-25T15:55:39.458 INFO:teuthology.orchestra.run.smithi076.stdout:(152/155): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-01-25T15:55:39.468 INFO:teuthology.orchestra.run.smithi033.stdout:(57/155): openssl-devel-3.0.7-25.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-01-25T15:55:39.521 INFO:teuthology.orchestra.run.smithi152.stdout:(63/155): postgresql-server-13.11-1.el9.x86_64. 12 MB/s | 5.7 MB 00:00 2024-01-25T15:55:39.543 INFO:teuthology.orchestra.run.smithi033.stdout:(58/155): postgresql-13.11-1.el9.x86_64.rpm 20 MB/s | 1.5 MB 00:00 2024-01-25T15:55:39.545 INFO:teuthology.orchestra.run.smithi152.stdout:(64/155): python3-attrs-20.3.0-7.el9.noarch.rpm 163 kB/s | 78 kB 00:00 2024-01-25T15:55:39.558 INFO:teuthology.orchestra.run.smithi076.stdout:(153/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-01-25T15:55:39.610 INFO:teuthology.orchestra.run.smithi033.stdout:(59/155): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-01-25T15:55:39.625 INFO:teuthology.orchestra.run.smithi076.stdout:(154/155): librbd1-19.0.0-885.g661946e7.el9.x86 2.8 MB/s | 3.1 MB 00:01 2024-01-25T15:55:39.660 INFO:teuthology.orchestra.run.smithi033.stdout:(60/155): postgresql-private-libs-13.11-1.el9.x 2.6 MB/s | 134 kB 00:00 2024-01-25T15:55:39.754 INFO:teuthology.orchestra.run.smithi152.stdout:(65/155): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2024-01-25T15:55:39.771 INFO:teuthology.orchestra.run.smithi152.stdout:(66/155): python3-jsonpatch-1.21-16.el9.noarch. 110 kB/s | 24 kB 00:00 2024-01-25T15:55:39.796 INFO:teuthology.orchestra.run.smithi152.stdout:(67/155): python3-jinja2-2.11.3-4.el9.noarch.rp 923 kB/s | 228 kB 00:00 2024-01-25T15:55:39.813 INFO:teuthology.orchestra.run.smithi152.stdout:(68/155): python3-jsonpointer-2.0-4.el9.noarch. 308 kB/s | 18 kB 00:00 2024-01-25T15:55:39.835 INFO:teuthology.orchestra.run.smithi033.stdout:(61/155): postgresql-server-13.11-1.el9.x86_64. 32 MB/s | 5.7 MB 00:00 2024-01-25T15:55:39.838 INFO:teuthology.orchestra.run.smithi152.stdout:(69/155): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2024-01-25T15:55:39.878 INFO:teuthology.orchestra.run.smithi033.stdout:(62/155): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-01-25T15:55:39.880 INFO:teuthology.orchestra.run.smithi152.stdout:(70/155): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 32 kB 00:00 2024-01-25T15:55:39.883 INFO:teuthology.orchestra.run.smithi076.stdout:(155/155): librados2-19.0.0-885.g661946e7.el9.x 2.3 MB/s | 3.4 MB 00:01 2024-01-25T15:55:39.888 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2024-01-25T15:55:39.888 INFO:teuthology.orchestra.run.smithi076.stdout:Total 11 MB/s | 209 MB 00:18 2024-01-25T15:55:39.905 INFO:teuthology.orchestra.run.smithi152.stdout:(71/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.2 MB/s | 217 kB 00:00 2024-01-25T15:55:39.916 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2024-01-25T15:55:40.106 INFO:teuthology.orchestra.run.smithi152.stdout:(72/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 4.8 MB 00:00 2024-01-25T15:55:40.119 INFO:teuthology.orchestra.run.smithi033.stdout:(63/155): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 5.8 MB 00:00 2024-01-25T15:55:40.131 INFO:teuthology.orchestra.run.smithi152.stdout:(73/155): python3-oauthlib-3.1.1-5.el9.noarch.r 697 kB/s | 174 kB 00:00 2024-01-25T15:55:40.156 INFO:teuthology.orchestra.run.smithi152.stdout:(74/155): python3-prettytable-0.7.2-27.el9.noar 165 kB/s | 41 kB 00:00 2024-01-25T15:55:40.161 INFO:teuthology.orchestra.run.smithi033.stdout:(64/155): python3-jinja2-2.11.3-4.el9.noarch.rp 5.4 MB/s | 228 kB 00:00 2024-01-25T15:55:40.166 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2024-01-25T15:55:40.166 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2024-01-25T15:55:40.182 INFO:teuthology.orchestra.run.smithi152.stdout:(75/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-01-25T15:55:40.207 INFO:teuthology.orchestra.run.smithi152.stdout:(76/155): python3-pytz-2021.1-5.el9.noarch.rpm 952 kB/s | 47 kB 00:00 2024-01-25T15:55:40.232 INFO:teuthology.orchestra.run.smithi152.stdout:(77/155): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-01-25T15:55:40.253 INFO:teuthology.orchestra.run.smithi033.stdout:(65/155): ceph-mgr-diskprediction-local-19.0.0- 2.6 MB/s | 7.4 MB 00:02 2024-01-25T15:55:40.257 INFO:teuthology.orchestra.run.smithi152.stdout:(78/155): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-01-25T15:55:40.270 INFO:teuthology.orchestra.run.smithi033.stdout:(66/155): python3-jsonpatch-1.21-16.el9.noarch. 225 kB/s | 24 kB 00:00 2024-01-25T15:55:40.287 INFO:teuthology.orchestra.run.smithi033.stdout:(67/155): python3-jsonpointer-2.0-4.el9.noarch. 541 kB/s | 18 kB 00:00 2024-01-25T15:55:40.396 INFO:teuthology.orchestra.run.smithi033.stdout:(68/155): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2024-01-25T15:55:40.463 INFO:teuthology.orchestra.run.smithi033.stdout:(69/155): python3-markupsafe-1.1.1-12.el9.x86_6 180 kB/s | 32 kB 00:00 2024-01-25T15:55:40.491 INFO:teuthology.orchestra.run.smithi152.stdout:(79/155): python3-toml-0.10.2-6.el9.noarch.rpm 167 kB/s | 39 kB 00:00 2024-01-25T15:55:40.516 INFO:teuthology.orchestra.run.smithi152.stdout:(80/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 160 kB/s | 41 kB 00:00 2024-01-25T15:55:40.580 INFO:teuthology.orchestra.run.smithi033.stdout:(70/155): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2024-01-25T15:55:40.607 INFO:teuthology.orchestra.run.smithi033.stdout:(71/155): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-01-25T15:55:40.817 INFO:teuthology.orchestra.run.smithi152.stdout:(81/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-01-25T15:55:40.842 INFO:teuthology.orchestra.run.smithi152.stdout:(82/155): socat-1.7.4.1-5.el9.x86_64.rpm 856 kB/s | 300 kB 00:00 2024-01-25T15:55:40.867 INFO:teuthology.orchestra.run.smithi152.stdout:(83/155): uuid-1.6.2-55.el9.x86_64.rpm 162 kB/s | 56 kB 00:00 2024-01-25T15:55:40.892 INFO:teuthology.orchestra.run.smithi152.stdout:(84/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-01-25T15:55:40.926 INFO:teuthology.orchestra.run.smithi152.stdout:(85/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 753 kB/s | 44 kB 00:00 2024-01-25T15:55:40.951 INFO:teuthology.orchestra.run.smithi152.stdout:(86/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 746 kB/s | 63 kB 00:00 2024-01-25T15:55:40.977 INFO:teuthology.orchestra.run.smithi152.stdout:(87/155): fuse-2.9.9-15.el9.x86_64.rpm 936 kB/s | 79 kB 00:00 2024-01-25T15:55:41.002 INFO:teuthology.orchestra.run.smithi152.stdout:(88/155): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-01-25T15:55:41.016 INFO:teuthology.orchestra.run.smithi033.stdout:(72/155): ceph-test-19.0.0-885.g661946e7.el9.x8 4.7 MB/s | 46 MB 00:09 2024-01-25T15:55:41.027 INFO:teuthology.orchestra.run.smithi152.stdout:(89/155): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 194 kB 00:00 2024-01-25T15:55:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:(73/155): python3-oauthlib-3.1.1-5.el9.noarch.r 379 kB/s | 174 kB 00:00 2024-01-25T15:55:41.066 INFO:teuthology.orchestra.run.smithi033.stdout:(74/155): python3-prettytable-0.7.2-27.el9.noar 90 kB/s | 41 kB 00:00 2024-01-25T15:55:41.069 INFO:teuthology.orchestra.run.smithi152.stdout:(90/155): libgfortran-11.4.1-3.el9.x86_64.rpm 8.5 MB/s | 802 kB 00:00 2024-01-25T15:55:41.091 INFO:teuthology.orchestra.run.smithi033.stdout:(75/155): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2024-01-25T15:55:41.094 INFO:teuthology.orchestra.run.smithi152.stdout:(91/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-01-25T15:55:41.117 INFO:teuthology.orchestra.run.smithi033.stdout:(76/155): python3-pytz-2021.1-5.el9.noarch.rpm 951 kB/s | 47 kB 00:00 2024-01-25T15:55:41.119 INFO:teuthology.orchestra.run.smithi152.stdout:(92/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-01-25T15:55:41.143 INFO:teuthology.orchestra.run.smithi033.stdout:(77/155): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-01-25T15:55:41.145 INFO:teuthology.orchestra.run.smithi152.stdout:(93/155): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-01-25T15:55:41.167 INFO:teuthology.orchestra.run.smithi033.stdout:(78/155): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-01-25T15:55:41.178 INFO:teuthology.orchestra.run.smithi152.stdout:(94/155): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-01-25T15:55:41.204 INFO:teuthology.orchestra.run.smithi152.stdout:(95/155): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2024-01-25T15:55:41.229 INFO:teuthology.orchestra.run.smithi152.stdout:(96/155): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2024-01-25T15:55:41.254 INFO:teuthology.orchestra.run.smithi152.stdout:(97/155): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-01-25T15:55:41.338 INFO:teuthology.orchestra.run.smithi152.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-01-25T15:55:41.384 INFO:teuthology.orchestra.run.smithi033.stdout:(79/155): python3-toml-0.10.2-6.el9.noarch.rpm 181 kB/s | 39 kB 00:00 2024-01-25T15:55:41.409 INFO:teuthology.orchestra.run.smithi033.stdout:(80/155): python3-wcwidth-0.2.5-8.el9.noarch.rp 171 kB/s | 41 kB 00:00 2024-01-25T15:55:41.421 INFO:teuthology.orchestra.run.smithi152.stdout:(99/155): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-01-25T15:55:41.577 INFO:teuthology.orchestra.run.smithi033.stdout:(81/155): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-01-25T15:55:41.602 INFO:teuthology.orchestra.run.smithi033.stdout:(82/155): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-01-25T15:55:41.627 INFO:teuthology.orchestra.run.smithi033.stdout:(83/155): uuid-1.6.2-55.el9.x86_64.rpm 263 kB/s | 56 kB 00:00 2024-01-25T15:55:41.652 INFO:teuthology.orchestra.run.smithi033.stdout:(84/155): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-01-25T15:55:41.677 INFO:teuthology.orchestra.run.smithi033.stdout:(85/155): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-01-25T15:55:41.702 INFO:teuthology.orchestra.run.smithi033.stdout:(86/155): zlib-devel-1.2.11-40.el9.x86_64.rpm 588 kB/s | 44 kB 00:00 2024-01-25T15:55:41.714 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2024-01-25T15:55:41.714 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2024-01-25T15:55:41.728 INFO:teuthology.orchestra.run.smithi033.stdout:(87/155): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-01-25T15:55:41.755 INFO:teuthology.orchestra.run.smithi033.stdout:(88/155): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-01-25T15:55:41.787 INFO:teuthology.orchestra.run.smithi033.stdout:(89/155): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-01-25T15:55:41.805 INFO:teuthology.orchestra.run.smithi152.stdout:(100/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 68 kB/s | 26 kB 00:00 2024-01-25T15:55:41.812 INFO:teuthology.orchestra.run.smithi033.stdout:(90/155): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 194 kB 00:00 2024-01-25T15:55:41.837 INFO:teuthology.orchestra.run.smithi033.stdout:(91/155): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-01-25T15:55:41.862 INFO:teuthology.orchestra.run.smithi033.stdout:(92/155): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.1 MB/s | 241 kB 00:00 2024-01-25T15:55:41.896 INFO:teuthology.orchestra.run.smithi033.stdout:(93/155): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2024-01-25T15:55:41.921 INFO:teuthology.orchestra.run.smithi033.stdout:(94/155): python3-pycparser-2.20-6.el9.noarch.r 2.2 MB/s | 124 kB 00:00 2024-01-25T15:55:41.930 INFO:teuthology.orchestra.run.smithi152.stdout:(101/155): gperftools-libs-2.9.1-2.el9.x86_64.r 457 kB/s | 309 kB 00:00 2024-01-25T15:55:41.946 INFO:teuthology.orchestra.run.smithi033.stdout:(95/155): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-25T15:55:41.955 INFO:teuthology.orchestra.run.smithi152.stdout:(102/155): liboath-2.6.7-2.el9.x86_64.rpm 324 kB/s | 49 kB 00:00 2024-01-25T15:55:41.972 INFO:teuthology.orchestra.run.smithi033.stdout:(96/155): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2024-01-25T15:55:41.997 INFO:teuthology.orchestra.run.smithi033.stdout:(97/155): python3-urllib3-1.26.5-5.el9.noarch.r 2.4 MB/s | 187 kB 00:00 2024-01-25T15:55:42.039 INFO:teuthology.orchestra.run.smithi152.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-01-25T15:55:42.072 INFO:teuthology.orchestra.run.smithi033.stdout:(98/155): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-01-25T15:55:42.081 INFO:teuthology.orchestra.run.smithi152.stdout:(104/155): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-01-25T15:55:42.223 INFO:teuthology.orchestra.run.smithi033.stdout:(99/155): python3-devel-3.9.17-2.el9.x86_64.rpm 981 kB/s | 245 kB 00:00 2024-01-25T15:55:42.248 INFO:teuthology.orchestra.run.smithi033.stdout:(100/155): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-01-25T15:55:42.273 INFO:teuthology.orchestra.run.smithi152.stdout:(105/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.5 MB/s | 838 kB 00:00 2024-01-25T15:55:42.298 INFO:teuthology.orchestra.run.smithi033.stdout:(101/155): liboath-2.6.7-2.el9.x86_64.rpm 975 kB/s | 49 kB 00:00 2024-01-25T15:55:42.332 INFO:teuthology.orchestra.run.smithi152.stdout:(106/155): libarrow-9.0.0-10.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:00 2024-01-25T15:55:42.340 INFO:teuthology.orchestra.run.smithi033.stdout:(102/155): libarrow-doc-9.0.0-10.el9.noarch.rpm 222 kB/s | 26 kB 00:00 2024-01-25T15:55:42.357 INFO:teuthology.orchestra.run.smithi152.stdout:(107/155): python3-asyncssh-2.13.2-3.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-01-25T15:55:42.374 INFO:teuthology.orchestra.run.smithi033.stdout:(103/155): libunwind-1.6.2-1.el9.x86_64.rpm 893 kB/s | 67 kB 00:00 2024-01-25T15:55:42.382 INFO:teuthology.orchestra.run.smithi152.stdout:(108/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 408 kB/s | 43 kB 00:00 2024-01-25T15:55:42.433 INFO:teuthology.orchestra.run.smithi152.stdout:(109/155): python3-cachetools-4.2.4-1.el9.noarc 321 kB/s | 32 kB 00:00 2024-01-25T15:55:42.458 INFO:teuthology.orchestra.run.smithi152.stdout:(110/155): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-01-25T15:55:42.492 INFO:teuthology.orchestra.run.smithi152.stdout:(111/155): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-01-25T15:55:42.499 INFO:teuthology.orchestra.run.smithi033.stdout:(104/155): libarrow-9.0.0-10.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-01-25T15:55:42.524 INFO:teuthology.orchestra.run.smithi033.stdout:(105/155): luarocks-3.9.2-1.el9.noarch.rpm 820 kB/s | 151 kB 00:00 2024-01-25T15:55:42.550 INFO:teuthology.orchestra.run.smithi033.stdout:(106/155): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-01-25T15:55:42.550 INFO:teuthology.orchestra.run.smithi152.stdout:(112/155): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-01-25T15:55:42.575 INFO:teuthology.orchestra.run.smithi152.stdout:(113/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-01-25T15:55:42.583 INFO:teuthology.orchestra.run.smithi033.stdout:(107/155): python3-asyncssh-2.13.2-3.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-01-25T15:55:42.600 INFO:teuthology.orchestra.run.smithi033.stdout:(108/155): python3-bcrypt-3.2.2-1.el9.x86_64.rp 575 kB/s | 43 kB 00:00 2024-01-25T15:55:42.601 INFO:teuthology.orchestra.run.smithi152.stdout:(114/155): python3-colorama-0.4.4-12.el9.noarch 323 kB/s | 35 kB 00:00 2024-01-25T15:55:42.617 INFO:teuthology.orchestra.run.smithi033.stdout:(109/155): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-01-25T15:55:42.634 INFO:teuthology.orchestra.run.smithi033.stdout:(110/155): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-01-25T15:55:42.659 INFO:teuthology.orchestra.run.smithi152.stdout:(115/155): python3-google-auth-2.26.2-1.el9.noa 1.8 MB/s | 202 kB 00:00 2024-01-25T15:55:42.676 INFO:teuthology.orchestra.run.smithi033.stdout:(111/155): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-01-25T15:55:42.685 INFO:teuthology.orchestra.run.smithi152.stdout:(116/155): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-01-25T15:55:42.701 INFO:teuthology.orchestra.run.smithi033.stdout:(112/155): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-01-25T15:55:42.710 INFO:teuthology.orchestra.run.smithi152.stdout:(117/155): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-01-25T15:55:42.727 INFO:teuthology.orchestra.run.smithi033.stdout:(113/155): python3-colorama-0.4.4-12.el9.noarch 705 kB/s | 35 kB 00:00 2024-01-25T15:55:42.752 INFO:teuthology.orchestra.run.smithi033.stdout:(114/155): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-01-25T15:55:42.760 INFO:teuthology.orchestra.run.smithi152.stdout:(118/155): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-01-25T15:55:42.780 INFO:teuthology.orchestra.run.smithi033.stdout:(115/155): python3-google-auth-2.26.2-1.el9.noa 2.6 MB/s | 202 kB 00:00 2024-01-25T15:55:42.786 INFO:teuthology.orchestra.run.smithi152.stdout:(119/155): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2024-01-25T15:55:42.803 INFO:teuthology.orchestra.run.smithi033.stdout:(116/155): python3-jaraco-8.2.1-3.el9.noarch.rp 141 kB/s | 11 kB 00:00 2024-01-25T15:55:42.811 INFO:teuthology.orchestra.run.smithi152.stdout:(120/155): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2024-01-25T15:55:42.836 INFO:teuthology.orchestra.run.smithi033.stdout:(117/155): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-01-25T15:55:42.884 INFO:teuthology.orchestra.run.smithi033.stdout:(118/155): python3-jaraco-collections-3.0.0-8.e 218 kB/s | 23 kB 00:00 2024-01-25T15:55:42.912 INFO:teuthology.orchestra.run.smithi152.stdout:(121/155): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-01-25T15:55:42.912 INFO:teuthology.orchestra.run.smithi033.stdout:(119/155): python3-jaraco-functools-3.5.0-2.el9 179 kB/s | 19 kB 00:00 2024-01-25T15:55:42.937 INFO:teuthology.orchestra.run.smithi152.stdout:(122/155): python3-logutils-0.3.5-21.el9.noarch 307 kB/s | 46 kB 00:00 2024-01-25T15:55:42.937 INFO:teuthology.orchestra.run.smithi033.stdout:(120/155): python3-jaraco-text-3.2.0-6.el9.noar 196 kB/s | 20 kB 00:00 2024-01-25T15:55:42.962 INFO:teuthology.orchestra.run.smithi152.stdout:(123/155): python3-more-itertools-8.12.0-2.el9. 524 kB/s | 79 kB 00:00 2024-01-25T15:55:42.970 INFO:teuthology.orchestra.run.smithi033.stdout:(121/155): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-01-25T15:55:42.996 INFO:teuthology.orchestra.run.smithi033.stdout:(122/155): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-01-25T15:55:43.012 INFO:teuthology.orchestra.run.smithi152.stdout:(124/155): python3-natsort-7.1.1-5.el9.noarch.r 576 kB/s | 58 kB 00:00 2024-01-25T15:55:43.029 INFO:teuthology.orchestra.run.smithi033.stdout:(123/155): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-01-25T15:55:43.054 INFO:teuthology.orchestra.run.smithi033.stdout:(124/155): python3-natsort-7.1.1-5.el9.noarch.r 688 kB/s | 58 kB 00:00 2024-01-25T15:55:43.071 INFO:teuthology.orchestra.run.smithi152.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-01-25T15:55:43.080 INFO:teuthology.orchestra.run.smithi033.stdout:(125/155): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-01-25T15:55:43.087 INFO:teuthology.orchestra.run.smithi152.stdout:(126/155): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-01-25T15:55:43.105 INFO:teuthology.orchestra.run.smithi033.stdout:(126/155): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-01-25T15:55:43.113 INFO:teuthology.orchestra.run.smithi152.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 895 kB/s | 90 kB 00:00 2024-01-25T15:55:43.130 INFO:teuthology.orchestra.run.smithi033.stdout:(127/155): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-01-25T15:55:43.156 INFO:teuthology.orchestra.run.smithi033.stdout:(128/155): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-01-25T15:55:43.181 INFO:teuthology.orchestra.run.smithi033.stdout:(129/155): python3-repoze-lru-0.7-16.el9.noarch 406 kB/s | 31 kB 00:00 2024-01-25T15:55:43.206 INFO:teuthology.orchestra.run.smithi033.stdout:(130/155): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-01-25T15:55:43.219 INFO:teuthology.orchestra.run.smithi152.stdout:(128/155): python3-pyperclip-1.8.2-2.el9.noarch 216 kB/s | 28 kB 00:00 2024-01-25T15:55:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:(131/155): python3-rsa-4.9-2.el9.noarch.rpm 878 kB/s | 59 kB 00:00 2024-01-25T15:55:43.238 INFO:teuthology.orchestra.run.smithi152.stdout:(129/155): python3-repoze-lru-0.7-16.el9.noarch 204 kB/s | 31 kB 00:00 2024-01-25T15:55:43.240 INFO:teuthology.orchestra.run.smithi033.stdout:(132/155): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-01-25T15:55:43.257 INFO:teuthology.orchestra.run.smithi033.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-01-25T15:55:43.264 INFO:teuthology.orchestra.run.smithi152.stdout:(130/155): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-01-25T15:55:43.291 INFO:teuthology.orchestra.run.smithi033.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-01-25T15:55:43.315 INFO:teuthology.orchestra.run.smithi152.stdout:(131/155): python3-rsa-4.9-2.el9.noarch.rpm 626 kB/s | 59 kB 00:00 2024-01-25T15:55:43.316 INFO:teuthology.orchestra.run.smithi033.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-01-25T15:55:43.341 INFO:teuthology.orchestra.run.smithi033.stdout:(136/155): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-01-25T15:55:43.356 INFO:teuthology.orchestra.run.smithi152.stdout:(132/155): python3-tempora-5.0.0-2.el9.noarch.r 390 kB/s | 36 kB 00:00 2024-01-25T15:55:43.359 INFO:teuthology.orchestra.run.smithi033.stdout:(137/155): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-01-25T15:55:43.373 INFO:teuthology.orchestra.run.smithi152.stdout:(133/155): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-01-25T15:55:43.383 INFO:teuthology.orchestra.run.smithi033.stdout:(138/155): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-01-25T15:55:43.423 INFO:teuthology.orchestra.run.smithi152.stdout:(134/155): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-01-25T15:55:43.426 INFO:teuthology.orchestra.run.smithi033.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-01-25T15:55:43.457 INFO:teuthology.orchestra.run.smithi152.stdout:(135/155): python3-websocket-client-1.2.3-2.el9 892 kB/s | 90 kB 00:00 2024-01-25T15:55:43.459 INFO:teuthology.orchestra.run.smithi033.stdout:(140/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-01-25T15:55:43.490 INFO:teuthology.orchestra.run.smithi152.stdout:(136/155): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-01-25T15:55:43.532 INFO:teuthology.orchestra.run.smithi152.stdout:(137/155): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-01-25T15:55:43.543 INFO:teuthology.orchestra.run.smithi033.stdout:(141/155): NetworkManager-1.45.10-1.el9.x86_64. 27 MB/s | 2.2 MB 00:00 2024-01-25T15:55:43.566 INFO:teuthology.orchestra.run.smithi152.stdout:(138/155): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-01-25T15:55:43.618 INFO:teuthology.orchestra.run.smithi033.stdout:(142/155): NetworkManager-libnm-1.45.10-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-01-25T15:55:43.643 INFO:teuthology.orchestra.run.smithi033.stdout:(143/155): NetworkManager-team-1.45.10-1.el9.x8 1.7 MB/s | 42 kB 00:00 2024-01-25T15:55:43.658 INFO:teuthology.orchestra.run.smithi152.stdout:(139/155): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-01-25T15:55:43.677 INFO:teuthology.orchestra.run.smithi033.stdout:(144/155): NetworkManager-tui-1.45.10-1.el9.x86 7.2 MB/s | 246 kB 00:00 2024-01-25T15:55:43.700 INFO:teuthology.orchestra.run.smithi152.stdout:(140/155): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-01-25T15:55:43.702 INFO:teuthology.orchestra.run.smithi033.stdout:(145/155): acl-2.3.1-4.el9.x86_64.rpm 2.7 MB/s | 69 kB 00:00 2024-01-25T15:55:43.728 INFO:teuthology.orchestra.run.smithi033.stdout:(146/155): elfutils-debuginfod-client-0.190-2.e 1.4 MB/s | 35 kB 00:00 2024-01-25T15:55:43.770 INFO:teuthology.orchestra.run.smithi033.stdout:(147/155): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-01-25T15:55:43.792 INFO:teuthology.orchestra.run.smithi152.stdout:(141/155): NetworkManager-1.45.10-1.el9.x86_64. 25 MB/s | 2.2 MB 00:00 2024-01-25T15:55:43.812 INFO:teuthology.orchestra.run.smithi033.stdout:(148/155): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-01-25T15:55:43.837 INFO:teuthology.orchestra.run.smithi033.stdout:(149/155): libacl-2.3.1-4.el9.x86_64.rpm 877 kB/s | 22 kB 00:00 2024-01-25T15:55:43.867 INFO:teuthology.orchestra.run.smithi152.stdout:(142/155): NetworkManager-libnm-1.45.10-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-01-25T15:55:43.895 INFO:teuthology.orchestra.run.smithi033.stdout:(150/155): openssl-3.0.7-25.el9.x86_64.rpm 20 MB/s | 1.2 MB 00:00 2024-01-25T15:55:43.925 INFO:teuthology.orchestra.run.smithi152.stdout:(143/155): librados2-19.0.0-885.g661946e7.el9.x 8.5 MB/s | 3.4 MB 00:00 2024-01-25T15:55:43.951 INFO:teuthology.orchestra.run.smithi152.stdout:(144/155): NetworkManager-team-1.45.10-1.el9.x8 503 kB/s | 42 kB 00:00 2024-01-25T15:55:43.976 INFO:teuthology.orchestra.run.smithi152.stdout:(145/155): NetworkManager-tui-1.45.10-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-01-25T15:55:43.979 INFO:teuthology.orchestra.run.smithi033.stdout:(151/155): librados2-19.0.0-885.g661946e7.el9.x 5.4 MB/s | 3.4 MB 00:00 2024-01-25T15:55:43.993 INFO:teuthology.orchestra.run.smithi152.stdout:(146/155): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-01-25T15:55:44.019 INFO:teuthology.orchestra.run.smithi152.stdout:(147/155): elfutils-debuginfod-client-0.190-2.e 845 kB/s | 35 kB 00:00 2024-01-25T15:55:44.029 INFO:teuthology.orchestra.run.smithi033.stdout:(152/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 16 MB/s | 2.1 MB 00:00 2024-01-25T15:55:44.044 INFO:teuthology.orchestra.run.smithi152.stdout:(148/155): elfutils-libelf-0.190-2.el9.x86_64.r 3.7 MB/s | 191 kB 00:00 2024-01-25T15:55:44.055 INFO:teuthology.orchestra.run.smithi033.stdout:(153/155): xfsdump-3.1.12-4.el9.x86_64.rpm 4.3 MB/s | 321 kB 00:00 2024-01-25T15:55:44.069 INFO:teuthology.orchestra.run.smithi152.stdout:(149/155): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 254 kB 00:00 2024-01-25T15:55:44.094 INFO:teuthology.orchestra.run.smithi152.stdout:(150/155): libacl-2.3.1-4.el9.x86_64.rpm 442 kB/s | 22 kB 00:00 2024-01-25T15:55:44.096 INFO:teuthology.orchestra.run.smithi033.stdout:(154/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-01-25T15:55:44.137 INFO:teuthology.orchestra.run.smithi152.stdout:(151/155): openssl-3.0.7-25.el9.x86_64.rpm 17 MB/s | 1.2 MB 00:00 2024-01-25T15:55:44.187 INFO:teuthology.orchestra.run.smithi152.stdout:(152/155): openssl-libs-3.0.7-25.el9.x86_64.rpm 24 MB/s | 2.1 MB 00:00 2024-01-25T15:55:44.212 INFO:teuthology.orchestra.run.smithi152.stdout:(153/155): xfsdump-3.1.12-4.el9.x86_64.rpm 4.3 MB/s | 321 kB 00:00 2024-01-25T15:55:44.247 INFO:teuthology.orchestra.run.smithi033.stdout:(155/155): librbd1-19.0.0-885.g661946e7.el9.x86 3.6 MB/s | 3.1 MB 00:00 2024-01-25T15:55:44.251 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-01-25T15:55:44.251 INFO:teuthology.orchestra.run.smithi033.stdout:Total 9.6 MB/s | 209 MB 00:21 2024-01-25T15:55:44.254 INFO:teuthology.orchestra.run.smithi152.stdout:(154/155): xfsprogs-6.3.0-1.el9.x86_64.rpm 16 MB/s | 1.1 MB 00:00 2024-01-25T15:55:44.277 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-01-25T15:55:44.355 INFO:teuthology.orchestra.run.smithi152.stdout:(155/155): librbd1-19.0.0-885.g661946e7.el9.x86 4.0 MB/s | 3.1 MB 00:00 2024-01-25T15:55:44.361 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-01-25T15:55:44.362 INFO:teuthology.orchestra.run.smithi152.stdout:Total 10 MB/s | 209 MB 00:19 2024-01-25T15:55:44.390 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-01-25T15:55:44.571 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-01-25T15:55:44.571 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-01-25T15:55:44.642 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-01-25T15:55:44.642 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-01-25T15:55:46.194 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-01-25T15:55:46.194 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-01-25T15:55:46.448 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-01-25T15:55:46.448 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-01-25T15:55:47.811 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2024-01-25T15:55:47.920 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-25T15:55:47.959 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-25T15:55:48.011 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-25T15:55:48.046 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-25T15:55:48.071 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-25T15:55:48.113 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-25T15:55:48.431 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-25T15:55:48.564 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:48.903 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:48.932 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:49.023 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:49.052 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:55:49.128 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-25T15:55:49.211 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-25T15:55:49.378 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-25T15:55:49.435 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-25T15:55:49.463 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:49.566 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:49.589 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:49.650 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:49.753 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-25T15:55:49.865 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:49.925 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:50.001 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:50.042 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:50.302 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:50.338 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-25T15:55:50.406 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-25T15:55:50.462 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/171 2024-01-25T15:55:50.500 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/171 2024-01-25T15:55:50.535 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/171 2024-01-25T15:55:50.575 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/171 2024-01-25T15:55:50.649 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/171 2024-01-25T15:55:50.683 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/171 2024-01-25T15:55:50.716 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/171 2024-01-25T15:55:50.744 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/171 2024-01-25T15:55:50.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/171 2024-01-25T15:55:50.849 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-25T15:55:50.892 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-25T15:55:50.921 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-25T15:55:50.991 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-25T15:55:51.068 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-25T15:55:51.203 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-25T15:55:51.255 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-25T15:55:51.778 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-25T15:55:51.859 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-25T15:55:51.891 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-25T15:55:51.940 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-25T15:55:51.978 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-25T15:55:52.019 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-25T15:55:52.146 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-25T15:55:52.195 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-25T15:55:52.242 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-01-25T15:55:52.305 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-25T15:55:52.340 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-25T15:55:52.341 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-25T15:55:52.380 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-25T15:55:52.394 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-25T15:55:52.433 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-25T15:55:52.458 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-25T15:55:52.476 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-25T15:55:52.491 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-25T15:55:52.501 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-25T15:55:52.527 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-25T15:55:52.544 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-25T15:55:52.574 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-25T15:55:52.610 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-25T15:55:52.653 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-25T15:55:52.704 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-25T15:55:52.837 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-25T15:55:52.850 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-01-25T15:55:52.877 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-25T15:55:52.884 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-25T15:55:52.914 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-25T15:55:52.955 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-25T15:55:52.959 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/171 2024-01-25T15:55:53.024 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/171 2024-01-25T15:55:53.027 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:53.049 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-25T15:55:53.097 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-25T15:55:53.138 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-25T15:55:53.164 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-25T15:55:53.165 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/171 2024-01-25T15:55:53.257 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-25T15:55:53.299 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 65/171 2024-01-25T15:55:53.308 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:53.310 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/171 2024-01-25T15:55:53.337 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/171 2024-01-25T15:55:53.337 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:53.348 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ply-3.11-14.el9.noarch 66/171 2024-01-25T15:55:53.400 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/171 2024-01-25T15:55:53.411 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 67/171 2024-01-25T15:55:53.428 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:53.458 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:55:53.547 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-25T15:55:53.594 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 68/171 2024-01-25T15:55:53.635 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-25T15:55:53.639 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 69/171 2024-01-25T15:55:53.765 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 70/171 2024-01-25T15:55:53.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 71/171 2024-01-25T15:55:53.803 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/171 2024-01-25T15:55:53.813 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-25T15:55:53.834 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/171 2024-01-25T15:55:53.836 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:53.866 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/171 2024-01-25T15:55:53.867 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-25T15:55:53.886 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:53.902 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-16.el9.x86_64 74/171 2024-01-25T15:55:53.933 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/171 2024-01-25T15:55:53.944 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:55:53.960 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/171 2024-01-25T15:55:53.971 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:53.973 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:53.995 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/171 2024-01-25T15:55:53.996 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:54.034 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/171 2024-01-25T15:55:54.040 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:54.047 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:55:54.066 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 79/171 2024-01-25T15:55:54.089 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:55:54.095 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:55:54.136 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-25T15:55:54.172 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 81/171 2024-01-25T15:55:54.192 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/171 2024-01-25T15:55:54.206 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 82/171 2024-01-25T15:55:54.225 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:54.239 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 83/171 2024-01-25T15:55:54.269 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:54.280 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 84/171 2024-01-25T15:55:54.283 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 12/171 2024-01-25T15:55:54.316 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 85/171 2024-01-25T15:55:54.344 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:54.401 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:54.404 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 86/171 2024-01-25T15:55:54.435 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 87/171 2024-01-25T15:55:54.479 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 13/171 2024-01-25T15:55:54.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.11-1.el9.x86_64 14/171 2024-01-25T15:55:54.536 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 88/171 2024-01-25T15:55:54.562 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:54.585 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 89/171 2024-01-25T15:55:54.655 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:54.659 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 15/171 2024-01-25T15:55:54.688 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:54.689 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-25T15:55:54.733 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 16/171 2024-01-25T15:55:54.757 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-25T15:55:54.811 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/171 2024-01-25T15:55:54.842 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/171 2024-01-25T15:55:54.877 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/171 2024-01-25T15:55:54.910 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 17/171 2024-01-25T15:55:54.917 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/171 2024-01-25T15:55:54.981 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/171 2024-01-25T15:55:55.009 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:55.025 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/171 2024-01-25T15:55:55.058 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/171 2024-01-25T15:55:55.069 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 18/171 2024-01-25T15:55:55.087 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/171 2024-01-25T15:55:55.156 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/171 2024-01-25T15:55:55.182 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:55.192 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-25T15:55:55.235 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-25T15:55:55.237 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:55.264 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-25T15:55:55.315 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-25T15:55:55.335 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-25T15:55:55.366 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 91/171 2024-01-25T15:55:55.400 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 92/171 2024-01-25T15:55:55.411 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-25T15:55:55.448 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 93/171 2024-01-25T15:55:55.507 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 19/171 2024-01-25T15:55:55.544 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-25T15:55:55.544 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/171 2024-01-25T15:55:55.547 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/171 2024-01-25T15:55:55.576 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/171 2024-01-25T15:55:55.589 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-25T15:55:55.620 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/171 2024-01-25T15:55:55.673 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 22/171 2024-01-25T15:55:55.704 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 23/171 2024-01-25T15:55:55.748 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 24/171 2024-01-25T15:55:55.755 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/171 2024-01-25T15:55:55.782 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/171 2024-01-25T15:55:55.787 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 25/171 2024-01-25T15:55:55.853 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 26/171 2024-01-25T15:55:55.860 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/171 2024-01-25T15:55:55.887 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 27/171 2024-01-25T15:55:55.920 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/171 2024-01-25T15:55:55.948 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 29/171 2024-01-25T15:55:56.019 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/171 2024-01-25T15:55:56.053 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 31/171 2024-01-25T15:55:56.097 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 32/171 2024-01-25T15:55:56.111 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-25T15:55:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 33/171 2024-01-25T15:55:56.195 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/171 2024-01-25T15:55:56.196 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-25T15:55:56.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-25T15:55:56.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/171 2024-01-25T15:55:56.276 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-25T15:55:56.314 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-25T15:55:56.356 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-25T15:55:56.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/171 2024-01-25T15:55:56.467 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/171 2024-01-25T15:55:56.487 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-25T15:55:56.515 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/171 2024-01-25T15:55:56.539 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-25T15:55:56.622 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/171 2024-01-25T15:55:56.647 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-25T15:55:56.676 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-25T15:55:56.731 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-25T15:55:56.803 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-25T15:55:56.835 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-25T15:55:56.871 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-25T15:55:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-25T15:55:56.962 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-25T15:55:57.005 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-25T15:55:57.020 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/171 2024-01-25T15:55:57.047 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-25T15:55:57.098 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/171 2024-01-25T15:55:57.140 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 40/171 2024-01-25T15:55:57.181 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-25T15:55:57.185 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 41/171 2024-01-25T15:55:57.220 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-25T15:55:57.223 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 42/171 2024-01-25T15:55:57.250 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-25T15:55:57.265 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/171 2024-01-25T15:55:57.291 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-25T15:55:57.386 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-25T15:55:57.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 44/171 2024-01-25T15:55:57.424 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-25T15:55:57.464 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-25T15:55:57.465 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 45/171 2024-01-25T15:55:57.489 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-25T15:55:57.576 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-3.el9.x86_64 46/171 2024-01-25T15:55:57.583 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-25T15:55:57.611 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 47/171 2024-01-25T15:55:57.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 65/171 2024-01-25T15:55:57.667 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/171 2024-01-25T15:55:57.674 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 66/171 2024-01-25T15:55:57.738 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 67/171 2024-01-25T15:55:57.739 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 49/171 2024-01-25T15:55:57.778 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 50/171 2024-01-25T15:55:57.814 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 51/171 2024-01-25T15:55:57.840 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/171 2024-01-25T15:55:57.862 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 52/171 2024-01-25T15:55:57.897 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 53/171 2024-01-25T15:55:57.919 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/171 2024-01-25T15:55:57.924 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 68/171 2024-01-25T15:55:57.941 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 54/171 2024-01-25T15:55:57.947 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/171 2024-01-25T15:55:57.971 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 69/171 2024-01-25T15:55:57.980 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/171 2024-01-25T15:55:57.983 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 55/171 2024-01-25T15:55:58.110 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 70/171 2024-01-25T15:55:58.120 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 56/171 2024-01-25T15:55:58.152 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 71/171 2024-01-25T15:55:58.163 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 57/171 2024-01-25T15:55:58.185 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/171 2024-01-25T15:55:58.193 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 58/171 2024-01-25T15:55:58.216 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/171 2024-01-25T15:55:58.236 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 59/171 2024-01-25T15:55:58.260 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-16.el9.x86_64 74/171 2024-01-25T15:55:58.287 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/171 2024-01-25T15:55:58.291 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/171 2024-01-25T15:55:58.318 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:55:58.319 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/171 2024-01-25T15:55:58.331 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/171 2024-01-25T15:55:58.353 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/171 2024-01-25T15:55:58.374 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 61/171 2024-01-25T15:55:58.392 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/171 2024-01-25T15:55:58.408 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:55:58.415 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 62/171 2024-01-25T15:55:58.423 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 79/171 2024-01-25T15:55:58.440 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 107/171 2024-01-25T15:55:58.443 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 63/171 2024-01-25T15:55:58.453 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:55:58.476 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/171 2024-01-25T15:55:58.529 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 81/171 2024-01-25T15:55:58.530 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 64/171 2024-01-25T15:55:58.564 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 82/171 2024-01-25T15:55:58.574 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 65/171 2024-01-25T15:55:58.598 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 83/171 2024-01-25T15:55:58.623 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 66/171 2024-01-25T15:55:58.639 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 84/171 2024-01-25T15:55:58.674 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 85/171 2024-01-25T15:55:58.688 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 67/171 2024-01-25T15:55:58.762 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 86/171 2024-01-25T15:55:58.801 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 87/171 2024-01-25T15:55:58.883 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 68/171 2024-01-25T15:55:58.903 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 88/171 2024-01-25T15:55:58.905 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/171 2024-01-25T15:55:58.929 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 69/171 2024-01-25T15:55:58.938 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:55:58.960 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 89/171 2024-01-25T15:55:59.009 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:55:59.032 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 111/171 2024-01-25T15:55:59.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 70/171 2024-01-25T15:55:59.118 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 71/171 2024-01-25T15:55:59.151 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 72/171 2024-01-25T15:55:59.194 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 73/171 2024-01-25T15:55:59.227 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.6-16.el9.x86_64 74/171 2024-01-25T15:55:59.258 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 75/171 2024-01-25T15:55:59.285 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 76/171 2024-01-25T15:55:59.328 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 77/171 2024-01-25T15:55:59.368 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 78/171 2024-01-25T15:55:59.399 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 79/171 2024-01-25T15:55:59.429 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:55:59.507 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 81/171 2024-01-25T15:55:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 82/171 2024-01-25T15:55:59.581 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 83/171 2024-01-25T15:55:59.623 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 84/171 2024-01-25T15:55:59.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 85/171 2024-01-25T15:55:59.692 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-25T15:55:59.740 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 91/171 2024-01-25T15:55:59.757 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 86/171 2024-01-25T15:55:59.777 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 92/171 2024-01-25T15:55:59.797 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 87/171 2024-01-25T15:55:59.822 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 93/171 2024-01-25T15:55:59.902 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 88/171 2024-01-25T15:55:59.909 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/171 2024-01-25T15:55:59.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/171 2024-01-25T15:55:59.957 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 89/171 2024-01-25T15:56:00.144 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/171 2024-01-25T15:56:00.172 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/171 2024-01-25T15:56:00.250 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/171 2024-01-25T15:56:00.708 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 90/171 2024-01-25T15:56:00.763 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 91/171 2024-01-25T15:56:00.800 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 92/171 2024-01-25T15:56:00.846 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 93/171 2024-01-25T15:56:00.893 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/171 2024-01-25T15:56:00.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 94/171 2024-01-25T15:56:00.974 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 95/171 2024-01-25T15:56:00.994 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/171 2024-01-25T15:56:01.125 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:01.154 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 96/171 2024-01-25T15:56:01.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 97/171 2024-01-25T15:56:01.267 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/171 2024-01-25T15:56:01.678 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:01.717 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:01.782 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:01.782 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-25T15:56:01.782 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-25T15:56:01.782 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:01.810 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:01.963 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/171 2024-01-25T15:56:02.070 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 100/171 2024-01-25T15:56:02.231 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/171 2024-01-25T15:56:02.307 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/171 2024-01-25T15:56:02.335 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/171 2024-01-25T15:56:02.360 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/171 2024-01-25T15:56:02.660 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/171 2024-01-25T15:56:02.689 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:56:02.780 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:56:02.810 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 107/171 2024-01-25T15:56:02.847 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/171 2024-01-25T15:56:03.280 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/171 2024-01-25T15:56:03.326 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:56:03.382 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 101/171 2024-01-25T15:56:03.397 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:56:03.427 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 111/171 2024-01-25T15:56:03.474 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 102/171 2024-01-25T15:56:03.502 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 103/171 2024-01-25T15:56:03.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 104/171 2024-01-25T15:56:03.874 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 105/171 2024-01-25T15:56:03.907 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:56:03.998 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 106/171 2024-01-25T15:56:04.028 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 107/171 2024-01-25T15:56:04.065 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 108/171 2024-01-25T15:56:04.516 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 109/171 2024-01-25T15:56:04.543 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:56:04.614 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 110/171 2024-01-25T15:56:04.636 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 111/171 2024-01-25T15:56:05.555 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:05.590 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:06.115 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:06.418 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:06.492 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:06.492 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-25T15:56:06.492 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-25T15:56:06.492 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:06.519 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 112/171 2024-01-25T15:56:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 113/171 2024-01-25T15:56:07.577 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-25T15:56:07.577 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-25T15:56:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /root 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2024-01-25T15:56:13.149 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:13.324 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:13.340 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:13.340 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:13.340 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-25T15:56:13.340 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:14.802 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:14.828 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-25T15:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:14.829 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:15.036 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:15.058 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-25T15:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:15.059 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:15.495 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:15.518 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:15.605 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.2-8.el9.noarch 119/171 2024-01-25T15:56:16.060 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:16.117 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-25T15:56:16.178 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-25T15:56:16.202 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:16.428 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:16.453 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:17.432 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:17.461 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:17.562 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:17.725 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-25T15:56:17.755 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:17.779 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:17.823 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:17.834 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:17.878 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-25T15:56:18.168 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-01-25T15:56:18.169 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:18.335 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:18.354 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:18.354 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:18.354 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-25T15:56:18.354 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:18.661 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/171 2024-01-25T15:56:18.691 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:18.714 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:18.759 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:18.780 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-25T15:56:18.781 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:18.835 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:18.857 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:19.168 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:19.191 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:19.812 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:19.838 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:19.838 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:19.839 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-25T15:56:19.839 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:19.839 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:19.839 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 114/171 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-01-25T15:56:19.951 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-01-25T15:56:19.952 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-01-25T15:56:19.952 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-01-25T15:56:19.952 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:20.047 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:20.069 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:20.069 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:20.069 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-25T15:56:20.069 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:20.070 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:20.070 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:20.124 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:20.139 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 115/171 2024-01-25T15:56:20.139 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:20.139 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-25T15:56:20.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:20.514 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:20.537 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:20.632 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 119/171 2024-01-25T15:56:21.010 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:21.038 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:21.068 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-25T15:56:21.129 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-25T15:56:21.153 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:21.377 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:21.404 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:21.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:21.685 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 116/171 2024-01-25T15:56:21.685 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:21.685 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-25T15:56:21.686 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:21.686 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-25T15:56:21.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:21.903 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 117/171 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-25T15:56:21.925 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:22.365 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:22.387 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:22.387 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 118/171 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-25T15:56:22.409 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:22.489 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:22.497 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 119/171 2024-01-25T15:56:22.649 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-25T15:56:22.706 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:22.729 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:22.729 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:22.729 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-25T15:56:22.730 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:22.730 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:22.730 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:22.773 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:22.785 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:22.829 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-25T15:56:22.960 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:22.987 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.0.0-885.g661946e7.el9.noarch 120/171 2024-01-25T15:56:23.017 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 121/171 2024-01-25T15:56:23.079 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 122/171 2024-01-25T15:56:23.101 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:23.333 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 123/171 2024-01-25T15:56:23.361 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:23.530 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-25T15:56:23.571 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-25T15:56:23.605 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/171 2024-01-25T15:56:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-25T15:56:23.641 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:23.661 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 138/171 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:23.664 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/171 2024-01-25T15:56:23.709 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:23.731 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:23.731 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:23.731 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-25T15:56:23.731 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:23.785 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:23.807 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:23.807 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:23.808 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-25T15:56:23.808 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:23.808 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:23.808 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:23.872 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:23.988 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 141/171 2024-01-25T15:56:24.019 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:24.051 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-25T15:56:24.083 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-25T15:56:24.113 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-25T15:56:24.135 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:24.158 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:24.177 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:24.228 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 147/171 2024-01-25T15:56:24.329 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 148/171 2024-01-25T15:56:24.371 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 124/171 2024-01-25T15:56:24.402 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:24.506 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 125/171 2024-01-25T15:56:24.672 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 126/171 2024-01-25T15:56:24.696 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:24.718 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 127/171 2024-01-25T15:56:24.718 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:24.719 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-25T15:56:24.719 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:24.719 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-25T15:56:24.719 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:24.764 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:24.774 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 128/171 2024-01-25T15:56:24.827 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.0.0-885.g661946e7.el9.x86_64 129/171 2024-01-25T15:56:24.867 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:24.910 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:24.937 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:24.991 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-25T15:56:25.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-45.el9.x86_64 130/171 2024-01-25T15:56:25.691 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 131/171 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-25T15:56:25.713 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:25.758 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:25.780 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-885.g661946 132/171 2024-01-25T15:56:25.780 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:25.780 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-25T15:56:25.780 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:25.825 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-25T15:56:25.825 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:25.847 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 133/171 2024-01-25T15:56:25.847 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:25.848 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-25T15:56:25.848 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:25.848 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-25T15:56:25.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:25.899 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-25T15:56:25.939 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 153/171 2024-01-25T15:56:26.020 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/171 2024-01-25T15:56:26.052 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/171 2024-01-25T15:56:26.077 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-25T15:56:26.111 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-25T15:56:26.138 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:26.176 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:26.198 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 134/171 2024-01-25T15:56:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-25T15:56:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-25T15:56:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-25T15:56:26.199 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:26.222 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:26.250 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 160/171 2024-01-25T15:56:26.282 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/171 2024-01-25T15:56:26.308 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/171 2024-01-25T15:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:26.333 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:26.401 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-25T15:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:26.612 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:26.695 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:26.720 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-25T15:56:26.746 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-25T15:56:26.788 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-25T15:56:26.851 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:28.447 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-25T15:56:28.487 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-25T15:56:28.529 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-25T15:56:28.561 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 138/171 2024-01-25T15:56:28.588 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/171 2024-01-25T15:56:28.764 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:28.881 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 141/171 2024-01-25T15:56:28.959 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:28.959 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-25T15:56:28.991 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-25T15:56:29.020 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-25T15:56:29.096 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:29.143 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 147/171 2024-01-25T15:56:29.194 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 148/171 2024-01-25T15:56:29.765 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:29.809 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:29.836 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:29.898 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-25T15:56:30.732 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-25T15:56:30.754 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 135/171 2024-01-25T15:56:30.796 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 136/171 2024-01-25T15:56:30.796 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-25T15:56:30.836 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 137/171 2024-01-25T15:56:30.837 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 153/171 2024-01-25T15:56:30.868 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 138/171 2024-01-25T15:56:30.896 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 139/171 2024-01-25T15:56:30.919 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/171 2024-01-25T15:56:30.957 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/171 2024-01-25T15:56:30.982 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-25T15:56:31.008 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:31.034 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-25T15:56:31.034 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:31.089 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:31.129 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:31.157 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 160/171 2024-01-25T15:56:31.195 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/171 2024-01-25T15:56:31.199 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 141/171 2024-01-25T15:56:31.221 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/171 2024-01-25T15:56:31.234 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:31.246 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:31.246 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:31.267 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 143/171 2024-01-25T15:56:31.299 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 144/171 2024-01-25T15:56:31.329 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 145/171 2024-01-25T15:56:31.331 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:31.356 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-25T15:56:31.362 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:31.362 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:31.397 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:31.454 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 147/171 2024-01-25T15:56:31.485 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 148/171 2024-01-25T15:56:31.550 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:31.550 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-25T15:56:31.560 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-25T15:56:31.561 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-25T15:56:31.562 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-25T15:56:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-25T15:56:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-25T15:56:31.575 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/171 2024-01-25T15:56:31.576 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/171 2024-01-25T15:56:31.577 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/171 2024-01-25T15:56:31.578 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 84/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-16.el9.x86_64 88/171 2024-01-25T15:56:31.579 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 91/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 92/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 93/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ply-3.11-14.el9.noarch 94/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 95/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 96/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 97/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 98/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-25T15:56:31.580 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-25T15:56:31.581 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-25T15:56:31.582 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 147/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 148/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/171 2024-01-25T15:56:31.583 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 150/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 151/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 152/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 153/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-4.el9.x86_64 154/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : acl-2.3.1-3.el9.x86_64 155/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 158/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 159/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 160/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 161/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 162/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:31.584 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 164/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 165/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 166/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 167/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 168/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 169/171 2024-01-25T15:56:31.585 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 170/171 2024-01-25T15:56:31.634 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:31.658 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-25T15:56:31.684 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-25T15:56:31.717 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-25T15:56:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:32.014 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:32.050 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:32.086 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 149/171 2024-01-25T15:56:32.140 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 150/171 2024-01-25T15:56:33.005 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 151/171 2024-01-25T15:56:33.071 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 152/171 2024-01-25T15:56:33.104 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 153/171 2024-01-25T15:56:33.212 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 154/171 2024-01-25T15:56:33.250 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 155/171 2024-01-25T15:56:33.275 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 156/171 2024-01-25T15:56:33.300 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:33.327 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 158/171 2024-01-25T15:56:33.327 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:33.403 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/171 2024-01-25T15:56:33.431 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 160/171 2024-01-25T15:56:33.464 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 161/171 2024-01-25T15:56:33.486 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/171 2024-01-25T15:56:33.512 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:33.512 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:33.613 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 164/171 2024-01-25T15:56:33.638 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 165/171 2024-01-25T15:56:33.645 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:33.645 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:33.824 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 166/171 2024-01-25T15:56:33.824 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:33.907 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 167/171 2024-01-25T15:56:33.932 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 168/171 2024-01-25T15:56:33.958 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 169/171 2024-01-25T15:56:34.000 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 170/171 2024-01-25T15:56:34.068 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:34.086 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 171/171 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: acl-2.3.1-4.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-25T15:56:34.087 INFO:teuthology.orchestra.run.smithi076.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.2-8.el9.noarch 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.7.4-5.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.088 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.089 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.4-9.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-25T15:56:34.090 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-16.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.091 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: lua-5.4.4-4.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-25T15:56:34.092 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-13.11-1.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-25T15:56:34.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-25T15:56:34.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ply-3.11-14.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-25T15:56:34.096 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.097 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: re2-1:20211101-3.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-01-25T15:56:34.098 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2024-01-25T15:56:34.664 DEBUG:teuthology.parallel:result is None 2024-01-25T15:56:36.666 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-25T15:56:36.667 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-25T15:56:36.668 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-25T15:56:36.669 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/171 2024-01-25T15:56:36.682 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/171 2024-01-25T15:56:36.683 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/171 2024-01-25T15:56:36.684 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/171 2024-01-25T15:56:36.685 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 84/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-16.el9.x86_64 88/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 91/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 92/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 93/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 94/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 95/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 96/171 2024-01-25T15:56:36.686 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 97/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 98/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-25T15:56:36.687 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-25T15:56:36.688 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-25T15:56:36.689 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 147/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 148/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 150/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 151/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 152/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 153/171 2024-01-25T15:56:36.690 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : acl-2.3.1-4.el9.x86_64 154/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : acl-2.3.1-3.el9.x86_64 155/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 158/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 159/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 160/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 161/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 162/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 164/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 165/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 166/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 167/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 168/171 2024-01-25T15:56:36.691 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 169/171 2024-01-25T15:56:36.692 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 170/171 2024-01-25T15:56:38.671 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 171/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.0.0-885.g661946e7.el9.x86_64 1/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 3/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 4/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-885.g661946 5/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 6/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 7/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 8/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 9/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 10/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 11/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 12/171 2024-01-25T15:56:38.672 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 13/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 14/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_6 15/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 16/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 17/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 18/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_ 19/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.0.0-885.g661946e7.el9.x86_64 20/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.0.0-885.g661946e7.el9 21/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.0.0-885.g661946e7.el9.x 22/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 23/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 24/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 25/171 2024-01-25T15:56:38.673 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 26/171 2024-01-25T15:56:38.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 27/171 2024-01-25T15:56:38.686 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 28/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 29/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-885.g661946e7.e 30/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noar 31/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.no 32/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-885.g6619 33/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9 34/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 35/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el 36/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 37/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.0.0-885.g661946e7.el9.noarch 38/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 39/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/171 2024-01-25T15:56:38.687 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/171 2024-01-25T15:56:38.688 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/171 2024-01-25T15:56:38.689 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 84/171 2024-01-25T15:56:38.690 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.6-16.el9.x86_64 88/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 91/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 92/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 93/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 94/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 95/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 96/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 97/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 98/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 99/171 2024-01-25T15:56:38.691 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 100/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 101/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 102/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 103/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 104/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 105/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 106/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 107/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 108/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 109/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 110/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 111/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 112/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 113/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 114/171 2024-01-25T15:56:38.692 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 115/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 116/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 117/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 118/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 119/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 120/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/171 2024-01-25T15:56:38.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.0.0-885.g661946e7.el9.x86_64 140/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.0.0-885.g661946e7.el9.x86_64 142/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/171 2024-01-25T15:56:38.694 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 146/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 147/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 148/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 150/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 151/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 152/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 153/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-4.el9.x86_64 154/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : acl-2.3.1-3.el9.x86_64 155/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 156/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 157/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 158/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 159/171 2024-01-25T15:56:38.695 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 160/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 161/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 162/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 163/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 164/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 165/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 166/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 167/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 168/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 169/171 2024-01-25T15:56:38.696 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 170/171 2024-01-25T15:56:39.188 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 171/171 2024-01-25T15:56:39.188 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: acl-2.3.1-4.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.189 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.190 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.191 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.6-16.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-25T15:56:39.192 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-25T15:56:39.193 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.11-1.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-25T15:56:39.194 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-25T15:56:39.195 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-25T15:56:39.196 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-25T15:56:39.197 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-25T15:56:39.198 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-25T15:56:39.199 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-25T15:56:39.200 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-25T15:56:39.200 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:56:39.200 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-01-25T15:56:39.828 DEBUG:teuthology.parallel:result is None 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 171/171 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: acl-2.3.1-4.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-25T15:56:41.226 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-25T15:56:41.227 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.228 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.0.0-885.g661946e7.el9.noarch 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-25T15:56:41.229 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: jq-1.6-16.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-25T15:56:41.230 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-25T15:56:41.231 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.11-1.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.232 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-25T15:56:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-25T15:56:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-25T15:56:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.0.0-885.g661946e7.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-3.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-25T15:56:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:56:41.238 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-01-25T15:56:41.687 DEBUG:teuthology.parallel:result is None 2024-01-25T15:56:41.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:41.825 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-25T15:56:41.848 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-885.g661946e7.el9 2024-01-25T15:56:41.848 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-25T15:56:41.848 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-25T15:56:41.850 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:42.092 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-25T15:56:42.120 INFO:teuthology.orchestra.run.smithi076.stdout:19.0.0-885.g661946e7.el9 2024-01-25T15:56:42.120 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-25T15:56:42.121 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-25T15:56:42.123 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:42.263 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-25T15:56:42.288 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-885.g661946e7.el9 2024-01-25T15:56:42.289 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-885.g661946e7.el9 2024-01-25T15:56:42.289 INFO:teuthology.task.install:The correct ceph version 19.0.0-885.g661946e7 is installed. 2024-01-25T15:56:42.290 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-25T15:56:42.290 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:56:42.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-25T15:56:42.328 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:56:42.328 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-25T15:56:42.362 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:56:42.362 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-25T15:56:42.396 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-25T15:56:42.396 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:56:42.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-01-25T15:56:42.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-25T15:56:42.497 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:56:42.497 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-01-25T15:56:42.527 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-25T15:56:42.596 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:56:42.597 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-01-25T15:56:42.628 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-25T15:56:42.699 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-25T15:56:42.699 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:56:42.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-25T15:56:42.730 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-25T15:56:42.799 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:56:42.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-25T15:56:42.828 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-25T15:56:42.897 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:56:42.897 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-25T15:56:42.928 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-25T15:56:42.998 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-25T15:56:42.998 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:56:42.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-01-25T15:56:43.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-25T15:56:43.099 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:56:43.100 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-01-25T15:56:43.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-25T15:56:43.195 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:56:43.196 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-01-25T15:56:43.226 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-25T15:56:43.298 INFO:teuthology.run_tasks:Running task cephadm... 2024-01-25T15:56:43.405 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-25T15:56:43.405 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:43.406 INFO:tasks.cephadm:Cluster fsid is 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:56:43.406 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-01-25T15:56:43.406 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.76', 'mon.c': '172.21.15.152'} 2024-01-25T15:56:43.406 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-01-25T15:56:43.406 INFO:tasks.cephadm:First mgr is x 2024-01-25T15:56:43.406 INFO:tasks.cephadm:Normalizing hostnames... 2024-01-25T15:56:43.407 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-01-25T15:56:43.438 DEBUG:teuthology.orchestra.run.smithi076:> sudo hostname $(hostname -s) 2024-01-25T15:56:43.467 DEBUG:teuthology.orchestra.run.smithi152:> sudo hostname $(hostname -s) 2024-01-25T15:56:43.498 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-01-25T15:56:43.499 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:43.718 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '661946e7044723130c03627323f6a879a9848cb5', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76233/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-885-g661946e7', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-885.g661946e7'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-01-25 07:26:33.077331', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240125.044502', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-01-25T15:56:43.958 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240125.044502, sha1 661946e7044723130c03627323f6a879a9848cb5 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:56:43.960 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm 2024-01-25T15:56:43.960 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:44.405 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 25 15:56 /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:44.406 DEBUG:teuthology.orchestra.run.smithi076:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:44.985 INFO:teuthology.orchestra.run.smithi076.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 25 15:56 /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:44.985 DEBUG:teuthology.orchestra.run.smithi152:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240125.044502/661946e7044723130c03627323f6a879a9848cb5/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:45.700 INFO:teuthology.orchestra.run.smithi152.stdout:-rw-r--r--. 1 ubuntu ubuntu 760779 Jan 25 15:56 /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:45.700 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:45.719 DEBUG:teuthology.orchestra.run.smithi076:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:45.737 DEBUG:teuthology.orchestra.run.smithi152:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-01-25T15:56:45.772 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 on all hosts... 2024-01-25T15:56:45.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-25T15:56:45.774 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-25T15:56:45.780 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 pull 2024-01-25T15:56:46.603 INFO:teuthology.orchestra.run.smithi152.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-25T15:56:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-25T15:56:46.685 INFO:teuthology.orchestra.run.smithi076.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-25T15:57:16.110 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T15:57:16.111 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T15:57:34.850 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-01-25T15:57:34.850 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-25T15:57:34.850 INFO:teuthology.orchestra.run.smithi076.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-25T15:57:34.851 INFO:teuthology.orchestra.run.smithi076.stdout: "repo_digests": [ 2024-01-25T15:57:34.851 INFO:teuthology.orchestra.run.smithi076.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-25T15:57:34.851 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-01-25T15:57:34.851 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_version": "ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev)", 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout: "image_id": "0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775", 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout: "repo_digests": [ 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06" 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout: ] 2024-01-25T15:57:38.931 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-01-25T15:57:38.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-01-25T15:57:38.990 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph 2024-01-25T15:57:39.025 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph 2024-01-25T15:57:39.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-01-25T15:57:39.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /etc/ceph 2024-01-25T15:57:39.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /etc/ceph 2024-01-25T15:57:39.141 INFO:tasks.cephadm:Writing seed config... 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [client] debug client = 20 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-01-25T15:57:39.142 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-01-25T15:57:39.143 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-01-25T15:57:39.144 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-01-25T15:57:39.145 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-01-25T15:57:39.145 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-01-25T15:57:39.145 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-01-25T15:57:39.145 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-01-25T15:57:39.145 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:57:39.145 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-01-25T15:57:39.161 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 55cf2542-bb9a-11ee-95b2-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-01-25T15:57:39.161 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service 2024-01-25T15:57:39.204 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x.service 2024-01-25T15:57:39.246 INFO:tasks.cephadm:Bootstrapping... 2024-01-25T15:57:39.246 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 -v bootstrap --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-01-25T15:57:39.444 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-01-25T15:57:39.444 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5', '-v', 'bootstrap', '--fsid', '55cf2542-bb9a-11ee-95b2-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-01-25T15:57:39.470 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.8.1 2024-01-25T15:57:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-01-25T15:57:39.471 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-01-25T15:57:39.495 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.8.1 2024-01-25T15:57:39.495 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-01-25T15:57:39.496 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-01-25T15:57:39.503 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-25T15:57:39.503 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-25T15:57:39.509 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-25T15:57:39.510 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:57:39.517 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-01-25T15:57:39.524 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-01-25T15:57:39.524 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-01-25T15:57:39.524 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-01-25T15:57:39.547 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.8.1 2024-01-25T15:57:39.547 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.8.1 is present 2024-01-25T15:57:39.548 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-01-25T15:57:39.548 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-01-25T15:57:39.554 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-01-25T15:57:39.554 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-01-25T15:57:39.560 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-01-25T15:57:39.560 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:57:39.567 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-01-25T15:57:39.573 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-01-25T15:57:39.573 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-01-25T15:57:39.573 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-01-25T15:57:39.573 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:39.573 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 139734355839200 on /run/cephadm/55cf2542-bb9a-11ee-95b2-87774f69a715.lock 2024-01-25T15:57:39.574 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 139734355839200 acquired on /run/cephadm/55cf2542-bb9a-11ee-95b2-87774f69a715.lock 2024-01-25T15:57:39.575 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-01-25T15:57:39.575 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-01-25T15:57:39.575 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-01-25T15:57:39.577 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-01-25T15:57:39.577 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-01-25T15:57:39.580 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-01-25T15:57:39.580 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-01-25T15:57:39.580 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-01-25T15:57:39.582 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-01-25T15:57:39.583 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-01-25T15:57:39.584 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-01-25T15:57:39.584 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-01-25T15:57:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-25T15:57:40.045 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5... 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:7526f7e228dfc90a8cd9a697e920af939ccb44066789c4e5461c5efe2a22fe5d 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:ea3366b8a90265ae3f6d3736cb0de4b52da4ffec919b46c6cb5bc5be1372efe8 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 2024-01-25T15:57:40.046 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-01-25T15:57:40.900 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-25T15:57:40.900 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev) 2024-01-25T15:57:40.901 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-01-25T15:57:41.839 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-01-25T15:57:41.839 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-01-25T15:57:42.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQD2hLJlmD1DBxAA6gUNfop3/sZNhOlMYhaInQ== 2024-01-25T15:57:43.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQD3hLJlJqcMChAAjO6Y19cjygKCtWnBBJtDmQ== 2024-01-25T15:57:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQD4hLJl/of4ARAAXtBlPGj8YQqTbeg4ivln2g== 2024-01-25T15:57:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-01-25T15:57:51.779 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:51.780 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-01-25T15:57:51.780 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:57:51.780 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-01-25T15:57:52.725 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-01-25T15:57:52.934 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-01-25T15:57:53.070 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-01-25T15:57:53.239 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target → /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target. 2024-01-25T15:57:53.240 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target → /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target. 2024-01-25T15:57:53.469 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a 2024-01-25T15:57:53.469 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service: Unit ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service not loaded. 2024-01-25T15:57:53.623 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target.wants/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service → /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@.service. 2024-01-25T15:57:54.462 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-25T15:57:54.462 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-01-25T15:57:54.470 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-25T15:57:54.470 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:57:54.470 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-01-25T15:57:54.470 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-01-25T15:57:54.471 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-01-25T15:57:54.471 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-01-25T15:57:55.704 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:55 smithi033 ceph-mon[31028]: from='client.? 172.21.15.33:0/4112563627' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-01-25T15:57:55.817 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-01-25T15:57:55.817 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:55.817 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-01-25T15:57:55.817 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:55.817 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.555567s) 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.922+0000 7fe1c576a700 1 Processor -- start 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.922+0000 7fe1c576a700 1 -- start start 2024-01-25T15:57:55.818 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.922+0000 7fe1c576a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.922+0000 7fe1c576a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c0107c20 con 0x7fe1c0107230 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.923+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.923+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53124/0 (socket says 172.21.15.33:53124) 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.923+0000 7fe1beffd700 1 -- 172.21.15.33:0/2126567449 learned_addr learned my addr 172.21.15.33:0/2126567449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.923+0000 7fe1beffd700 1 -- 172.21.15.33:0/2126567449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0108430 con 0x7fe1c0107230 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.923+0000 7fe1beffd700 1 --2- 172.21.15.33:0/2126567449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00098a0 tx=0x7fe1b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676f6568e719bdb2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.924+0000 7fe1be7fc700 1 -- 172.21.15.33:0/2126567449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b0004160 con 0x7fe1c0107230 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.924+0000 7fe1be7fc700 1 -- 172.21.15.33:0/2126567449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b00042c0 con 0x7fe1c0107230 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.924+0000 7fe1be7fc700 1 -- 172.21.15.33:0/2126567449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b00045a0 con 0x7fe1c0107230 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.924+0000 7fe1c576a700 1 -- 172.21.15.33:0/2126567449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 msgr2=0x7fe1c0107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.924+0000 7fe1c576a700 1 --2- 172.21.15.33:0/2126567449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00098a0 tx=0x7fe1b0009bb0 comp rx=0 tx=0).stop 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 -- 172.21.15.33:0/2126567449 shutdown_connections 2024-01-25T15:57:55.819 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 --2- 172.21.15.33:0/2126567449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c0107650 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 -- 172.21.15.33:0/2126567449 >> 172.21.15.33:0/2126567449 conn(0x7fe1c0102890 msgr2=0x7fe1c0104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 -- 172.21.15.33:0/2126567449 shutdown_connections 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 -- 172.21.15.33:0/2126567449 wait complete. 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.925+0000 7fe1c576a700 1 Processor -- start 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1c576a700 1 -- start start 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1c576a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1c576a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c019ac50 con 0x7fe1c0107230 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1beffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53138/0 (socket says 172.21.15.33:53138) 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1beffd700 1 -- 172.21.15.33:0/4112563627 learned_addr learned my addr 172.21.15.33:0/4112563627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.926+0000 7fe1beffd700 1 -- 172.21.15.33:0/4112563627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b0009550 con 0x7fe1c0107230 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1beffd700 1 --2- 172.21.15.33:0/4112563627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0004000 tx=0x7fe1b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:55.820 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b0011630 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0011790 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c019ae50 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1b0011a30 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c019b2f0 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe1b0021070 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.927+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe1b001b810 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.928+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c01943b0 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:54.932+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fe1b001ba70 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.006+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fe1c01089a0 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.007+0000 7fe1bcff9700 1 -- 172.21.15.33:0/4112563627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fe1b001bc10 con 0x7fe1c0107230 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 msgr2=0x7fe1c019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 --2- 172.21.15.33:0/4112563627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0004000 tx=0x7fe1b0004750 comp rx=0 tx=0).stop 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 shutdown_connections 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 --2- 172.21.15.33:0/4112563627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1c0107230 0x7fe1c019a710 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:55.821 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 >> 172.21.15.33:0/4112563627 conn(0x7fe1c0102890 msgr2=0x7fe1c01914f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:55.822 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 shutdown_connections 2024-01-25T15:57:55.822 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:55.008+0000 7fe1c576a700 1 -- 172.21.15.33:0/4112563627 wait complete. 2024-01-25T15:57:55.822 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-01-25T15:57:55.822 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-01-25T15:57:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:56 smithi033 ceph-mon[31028]: from='client.? 172.21.15.33:0/3205679750' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-01-25T15:57:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:56 smithi033 ceph-mon[31028]: from='client.? 172.21.15.33:0/3205679750' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-25T15:57:57.276 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.232+0000 7f58bbbbd700 1 Processor -- start 2024-01-25T15:57:57.277 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58bbbbd700 1 -- start start 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58bbbbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58bbbbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4107c90 con 0x7f58b41072a0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58b9959700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58b9959700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53152/0 (socket says 172.21.15.33:53152) 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58b9959700 1 -- 172.21.15.33:0/579872242 learned_addr learned my addr 172.21.15.33:0/579872242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.233+0000 7f58b9959700 1 -- 172.21.15.33:0/579872242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4108400 con 0x7f58b41072a0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58b9959700 1 --2- 172.21.15.33:0/579872242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009e80 tx=0x7f58a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a8f563ec7c3da server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58b8957700 1 -- 172.21.15.33:0/579872242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a400e070 con 0x7f58b41072a0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58b8957700 1 -- 172.21.15.33:0/579872242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4003a20 con 0x7f58b41072a0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58b8957700 1 -- 172.21.15.33:0/579872242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a4003ce0 con 0x7f58b41072a0 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 -- 172.21.15.33:0/579872242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 msgr2=0x7f58b41076c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:57.278 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 --2- 172.21.15.33:0/579872242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f58a4009e80 tx=0x7f58a400b330 comp rx=0 tx=0).stop 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 -- 172.21.15.33:0/579872242 shutdown_connections 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 --2- 172.21.15.33:0/579872242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b41076c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 -- 172.21.15.33:0/579872242 >> 172.21.15.33:0/579872242 conn(0x7f58b41028a0 msgr2=0x7f58b4104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 -- 172.21.15.33:0/579872242 shutdown_connections 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.234+0000 7f58bbbbd700 1 -- 172.21.15.33:0/579872242 wait complete. 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58bbbbd700 1 Processor -- start 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58bbbbd700 1 -- start start 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58bbbbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58bbbbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b419ac40 con 0x7f58b41072a0 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58b9959700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58b9959700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53154/0 (socket says 172.21.15.33:53154) 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58b9959700 1 -- 172.21.15.33:0/3205679750 learned_addr learned my addr 172.21.15.33:0/3205679750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.235+0000 7f58b9959700 1 -- 172.21.15.33:0/3205679750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a4009b30 con 0x7f58b41072a0 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b9959700 1 --2- 172.21.15.33:0/3205679750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f58a4004000 tx=0x7f58a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:57.279 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a4004370 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4004510 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b419ae40 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58a40047f0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b419b2e0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f58a40127c0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.236+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f58a401dbf0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.237+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b41943d0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.239+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f58a402b060 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.309+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f58b41089e0 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.312+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f58a4021030 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.313+0000 7f58b2ffd700 1 -- 172.21.15.33:0/3205679750 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f58a401de50 con 0x7f58b41072a0 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 msgr2=0x7f58b419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:57.280 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 --2- 172.21.15.33:0/3205679750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f58a4004000 tx=0x7f58a4004ad0 comp rx=0 tx=0).stop 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 shutdown_connections 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 --2- 172.21.15.33:0/3205679750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58b41072a0 0x7f58b419a700 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 >> 172.21.15.33:0/3205679750 conn(0x7f58b41028a0 msgr2=0x7f58b4191510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 shutdown_connections 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:56.314+0000 7f58bbbbd700 1 -- 172.21.15.33:0/3205679750 wait complete. 2024-01-25T15:57:57.281 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-01-25T15:57:58.062 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:57 smithi033 ceph-mon[31028]: from='client.? 172.21.15.33:0/4248631400' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:57:58.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.688+0000 7f207ea50700 1 Processor -- start 2024-01-25T15:57:58.320 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.688+0000 7f207ea50700 1 -- start start 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.688+0000 7f207ea50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.688+0000 7f207ea50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2078106b30 con 0x7f2078106140 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2077fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2077fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53162/0 (socket says 172.21.15.33:53162) 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2077fff700 1 -- 172.21.15.33:0/2823793037 learned_addr learned my addr 172.21.15.33:0/2823793037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2077fff700 1 -- 172.21.15.33:0/2823793037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078107340 con 0x7f2078106140 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2077fff700 1 --2- 172.21.15.33:0/2823793037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20600098a0 tx=0x7f2060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f904b2c262d9054a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2076ffd700 1 -- 172.21.15.33:0/2823793037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060004160 con 0x7f2078106140 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2076ffd700 1 -- 172.21.15.33:0/2823793037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f20600042c0 con 0x7f2078106140 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.689+0000 7f2076ffd700 1 -- 172.21.15.33:0/2823793037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060011a00 con 0x7f2078106140 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- 172.21.15.33:0/2823793037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 msgr2=0x7f2078106560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 --2- 172.21.15.33:0/2823793037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f20600098a0 tx=0x7f2060009bb0 comp rx=0 tx=0).stop 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- 172.21.15.33:0/2823793037 shutdown_connections 2024-01-25T15:57:58.321 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 --2- 172.21.15.33:0/2823793037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f2078106560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- 172.21.15.33:0/2823793037 >> 172.21.15.33:0/2823793037 conn(0x7f20781016e0 msgr2=0x7f2078103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- 172.21.15.33:0/2823793037 shutdown_connections 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- 172.21.15.33:0/2823793037 wait complete. 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 Processor -- start 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.690+0000 7f207ea50700 1 -- start start 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f207ea50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f207ea50700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207819b540 con 0x7f2078106140 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f2077fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f2077fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53178/0 (socket says 172.21.15.33:53178) 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f2077fff700 1 -- 172.21.15.33:0/4248631400 learned_addr learned my addr 172.21.15.33:0/4248631400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f2077fff700 1 -- 172.21.15.33:0/4248631400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060009550 con 0x7f2078106140 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f2077fff700 1 --2- 172.21.15.33:0/4248631400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2060011710 tx=0x7f2060011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060003aa0 con 0x7f2078106140 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2060003c00 con 0x7f2078106140 2024-01-25T15:57:58.322 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2060024800 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.691+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f207819b740 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.692+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207819bbe0 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.692+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2060024960 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.692+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f20600193c0 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.693+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20781950b0 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.695+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f2060024d50 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.765+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f207802d530 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.765+0000 7f20757fa700 1 -- 172.21.15.33:0/4248631400 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f206001abd0 con 0x7f2078106140 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 msgr2=0x7f207819b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 --2- 172.21.15.33:0/4248631400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2060011710 tx=0x7f2060011ad0 comp rx=0 tx=0).stop 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 shutdown_connections 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 --2- 172.21.15.33:0/4248631400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2078106140 0x7f207819b000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 >> 172.21.15.33:0/4248631400 conn(0x7f20781016e0 msgr2=0x7f2078102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 shutdown_connections 2024-01-25T15:57:58.323 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:57:57.766+0000 7f207ea50700 1 -- 172.21.15.33:0/4248631400 wait complete. 2024-01-25T15:57:58.324 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-01-25T15:57:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 systemd[1]: Stopping Ceph mon.a for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T15:57:58.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a[31024]: 2024-01-25T15:57:58.572+0000 7f50ab758700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T15:57:58.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a[31024]: 2024-01-25T15:57:58.572+0000 7f50ab758700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-25T15:57:58.900 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 podman[31287]: 2024-01-25 15:57:58.716197932 +0000 UTC m=+0.233019292 container died 9d555bada5ba58a6e1b2298460c08a70d6abd9320fe11f69bf73f8a30c1ddfa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-25T15:57:59.199 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 podman[31287]: 2024-01-25 15:57:58.988583473 +0000 UTC m=+0.505404787 container cleanup 9d555bada5ba58a6e1b2298460c08a70d6abd9320fe11f69bf73f8a30c1ddfa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-01-25T15:57:59.200 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:58 smithi033 bash[31287]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a 2024-01-25T15:57:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 podman[31302]: 2024-01-25 15:57:59.197846104 +0000 UTC m=+0.474665611 container remove 9d555bada5ba58a6e1b2298460c08a70d6abd9320fe11f69bf73f8a30c1ddfa3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T15:57:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service: Deactivated successfully. 2024-01-25T15:57:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 systemd[1]: Stopped Ceph mon.a for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T15:57:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 systemd[1]: Starting Ceph mon.a for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T15:57:59.881 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 podman[31398]: 2024-01-25 15:57:59.533054964 +0000 UTC m=+0.017707536 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 podman[31398]: 2024-01-25 15:57:59.631921297 +0000 UTC m=+0.116573871 container create 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 podman[31398]: 2024-01-25 15:57:59.800605328 +0000 UTC m=+0.285257894 container init 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3) 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 podman[31398]: 2024-01-25 15:57:59.803341898 +0000 UTC m=+0.287994456 container start 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: set uid:gid to 167:167 (ceph:ceph) 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: ceph version 19.0.0-885-g661946e7 (661946e7044723130c03627323f6a879a9848cb5) squid (dev), process ceph-mon, pid 2 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: pidfile_write: ignore empty --pid-file 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: load: jerasure load: lrc 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: RocksDB version: 7.9.2 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Git sha 0 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Compile date 2024-01-25 05:03:27 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: DB SUMMARY 2024-01-25T15:57:59.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: DB Session ID: 22QOOW7CU0XBOSDC6DGA 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: CURRENT file: CURRENT 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: IDENTITY file: IDENTITY 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 101170 ; 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.error_if_exists: 0 2024-01-25T15:57:59.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.create_if_missing: 0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.paranoid_checks: 1 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.flush_verify_memtable_count: 1 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.env: 0x56041515bae0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.fs: PosixFileSystem 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.info_log: 0x560415b01780 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_file_opening_threads: 16 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.statistics: (nil) 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.use_fsync: 0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_log_file_size: 0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.log_file_time_to_roll: 0 2024-01-25T15:57:59.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.keep_log_file_num: 1000 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.recycle_log_file_num: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_fallocate: 1 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_mmap_reads: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_mmap_writes: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.use_direct_reads: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.create_missing_column_families: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.db_log_dir: 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.wal_dir: 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.table_cache_numshardbits: 6 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.WAL_ttl_seconds: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.WAL_size_limit_MB: 0 2024-01-25T15:57:59.910 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.is_fd_close_on_exec: 1 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.advise_random_on_open: 1 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.db_write_buffer_size: 0 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.write_buffer_manager: 0x560415ad5400 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.use_adaptive_mutex: 0 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.rate_limiter: (nil) 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.wal_recovery_mode: 2 2024-01-25T15:57:59.911 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enable_thread_tracking: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enable_pipelined_write: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.unordered_write: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.row_cache: None 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.wal_filter: None 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_ingest_behind: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.two_write_queues: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.manual_wal_flush: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.wal_compression: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.atomic_flush: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-01-25T15:57:59.912 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.persist_stats_to_disk: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.write_dbid_to_manifest: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.log_readahead_size: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.best_efforts_recovery: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.allow_data_in_errors: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.db_host_id: __hostname__ 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enforce_single_del_contracts: true 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_background_jobs: 2 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_background_compactions: -1 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_subcompactions: 1 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.delayed_write_rate : 16777216 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_total_wal_size: 0 2024-01-25T15:57:59.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-01-25T15:57:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.stats_dump_period_sec: 600 2024-01-25T15:57:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.stats_persist_period_sec: 600 2024-01-25T15:57:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-01-25T15:57:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_open_files: -1 2024-01-25T15:57:59.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bytes_per_sync: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.wal_bytes_per_sync: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.strict_bytes_per_sync: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_readahead_size: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_background_flushes: -1 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Compression algorithms supported: 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kZSTD supported: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kXpressCompression supported: 0 2024-01-25T15:57:59.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kLZ4HCCompression supported: 1 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kZlibCompression supported: 1 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kSnappyCompression supported: 1 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kLZ4Compression supported: 1 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: kBZip2Compression supported: 0 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Fast CRC32 supported: Supported on x86 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: DMutex implementation: pthread_mutex_t 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.merge_operator: 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_filter: None 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_filter_factory: None 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.sst_partitioner_factory: None 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.memtable_factory: SkipListFactory 2024-01-25T15:57:59.916 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.table_factory: BlockBasedTable 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560415b018a0) 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x560415b05090 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-01-25T15:57:59.917 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-01-25T15:57:59.918 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.write_buffer_size: 33554432 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_write_buffer_number: 2 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression: NoCompression 2024-01-25T15:57:59.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression: Disabled 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.prefix_extractor: nullptr 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.num_levels: 7 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-01-25T15:57:59.920 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.window_bits: -14 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.level: 32767 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.strategy: 0 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.enabled: false 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-01-25T15:57:59.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.target_file_size_base: 67108864 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.target_file_size_multiplier: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-01-25T15:57:59.922 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.arena_block_size: 1048576 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.disable_auto_compactions: 0 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.table_properties_collectors: 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.inplace_update_support: 0 2024-01-25T15:57:59.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.inplace_update_num_locks: 10000 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.memtable_huge_page_size: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.bloom_locality: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.max_successive_merges: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.optimize_filters_for_hits: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.paranoid_file_checks: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.force_consistency_checks: 1 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.report_bg_io_stats: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.ttl: 2592000 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.periodic_compaction_seconds: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enable_blob_files: false 2024-01-25T15:57:59.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.min_blob_size: 0 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_file_size: 268435456 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_compression_type: NoCompression 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.enable_blob_garbage_collection: false 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.blob_file_starting_level: 0 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c679c914-7057-47e4-88e2-bbed17d488c9 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706198279831684, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-01-25T15:57:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706198279832706, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 96227, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 94215, "index_size": 294, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 14132, "raw_average_key_size": 50, "raw_value_size": 86529, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1706198279, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c679c914-7057-47e4-88e2-bbed17d488c9", "db_session_id": "22QOOW7CU0XBOSDC6DGA", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706198279832790, "job": 1, "event": "recovery_finished"} 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560415bf2000 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: DB pointer 0x560415be0000 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-01-25T15:57:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 95.81 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 95.81 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 140.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-01-25T15:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 30.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 30.92 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x560415b05090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???) e1 preinit fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).mds e1 new map 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).mds e1 print_map 2024-01-25T15:57:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 bash[31398]: 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 systemd[1]: Started Ceph mon.a for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mon.a is new leader, mons a in quorum (ranks 0) 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: monmap e1: 1 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: fsmap 2024-01-25T15:57:59.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: osdmap e1: 0 total, 0 up, 0 in 2024-01-25T15:57:59.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:57:59 smithi033 ceph-mon[31417]: mgrmap e1: no daemons active 2024-01-25T15:58:00.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6422dcb700 1 Processor -- start 2024-01-25T15:58:00.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6422dcb700 1 -- start start 2024-01-25T15:58:00.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6422dcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6422dcb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c107c70 con 0x7f641c107280 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6420b67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6420b67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57434/0 (socket says 172.21.15.33:57434) 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.299+0000 7f6420b67700 1 -- 172.21.15.33:0/3049646882 learned_addr learned my addr 172.21.15.33:0/3049646882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6420b67700 1 -- 172.21.15.33:0/3049646882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c1083e0 con 0x7f641c107280 2024-01-25T15:58:00.921 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6420b67700 1 --2- 172.21.15.33:0/3049646882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64100098a0 tx=0x7f6410009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2081d0235308b7c2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f641b7fe700 1 -- 172.21.15.33:0/3049646882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6410004160 con 0x7f641c107280 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f641b7fe700 1 -- 172.21.15.33:0/3049646882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f64100042c0 con 0x7f641c107280 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f641b7fe700 1 -- 172.21.15.33:0/3049646882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6410011a00 con 0x7f641c107280 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6422dcb700 1 -- 172.21.15.33:0/3049646882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 msgr2=0x7f641c1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6422dcb700 1 --2- 172.21.15.33:0/3049646882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64100098a0 tx=0x7f6410009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:00.922 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6422dcb700 1 -- 172.21.15.33:0/3049646882 shutdown_connections 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6422dcb700 1 --2- 172.21.15.33:0/3049646882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c1076a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.300+0000 7f6422dcb700 1 -- 172.21.15.33:0/3049646882 >> 172.21.15.33:0/3049646882 conn(0x7f641c102880 msgr2=0x7f641c104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 -- 172.21.15.33:0/3049646882 shutdown_connections 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 -- 172.21.15.33:0/3049646882 wait complete. 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 Processor -- start 2024-01-25T15:58:00.923 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 -- start start 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6422dcb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c07d0c0 con 0x7f641c107280 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6420b67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6420b67700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57442/0 (socket says 172.21.15.33:57442) 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.301+0000 7f6420b67700 1 -- 172.21.15.33:0/3265482204 learned_addr learned my addr 172.21.15.33:0/3265482204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:00.924 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6420b67700 1 -- 172.21.15.33:0/3265482204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6410009550 con 0x7f641c107280 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6420b67700 1 --2- 172.21.15.33:0/3265482204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6410009130 tx=0x7f6410011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6410003ab0 con 0x7f641c107280 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f6410003c10 con 0x7f641c107280 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c07b2b0 con 0x7f641c107280 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64100247d0 con 0x7f641c107280 2024-01-25T15:58:00.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c07b750 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6410024a50 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.302+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6410019410 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.303+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c062e10 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.306+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f641001abb0 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.375+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f641c07b990 con 0x7f641c107280 2024-01-25T15:58:00.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.379+0000 7f6419ffb700 1 -- 172.21.15.33:0/3265482204 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f641001ad50 con 0x7f641c107280 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 msgr2=0x7f641c07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 --2- 172.21.15.33:0/3265482204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6410009130 tx=0x7f6410011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 shutdown_connections 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 --2- 172.21.15.33:0/3265482204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c107280 0x7f641c07cb80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 >> 172.21.15.33:0/3265482204 conn(0x7f641c102880 msgr2=0x7f641c103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:00.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 shutdown_connections 2024-01-25T15:58:00.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:00.380+0000 7f6422dcb700 1 -- 172.21.15.33:0/3265482204 wait complete. 2024-01-25T15:58:00.928 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-01-25T15:58:00.933 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-01-25T15:58:00.934 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-01-25T15:58:00.934 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-01-25T15:58:00.935 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-01-25T15:58:01.161 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x 2024-01-25T15:58:01.161 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x.service: Unit ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x.service not loaded. 2024-01-25T15:58:01.310 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715.target.wants/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@.service. 2024-01-25T15:58:01.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:01 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3265482204' entity='client.admin' 2024-01-25T15:58:01.815 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:01 smithi033 podman[31631]: 2024-01-25 15:58:01.772049314 +0000 UTC m=+0.348525215 container start ea9ea84c77890c80f2551c4e9e8e53bb6c60d5e9fbff031080425527201701a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD) 2024-01-25T15:58:01.868 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-25T15:58:01.868 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-01-25T15:58:01.875 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-25T15:58:01.875 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:58:01.875 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-01-25T15:58:01.876 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-01-25T15:58:01.884 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-25T15:58:01.884 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-01-25T15:58:01.890 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-25T15:58:01.890 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:58:01.891 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-01-25T15:58:01.891 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-01-25T15:58:01.891 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-01-25T15:58:01.891 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-01-25T15:58:02.183 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:01 smithi033 bash[31631]: ea9ea84c77890c80f2551c4e9e8e53bb6c60d5e9fbff031080425527201701a3 2024-01-25T15:58:02.183 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:01 smithi033 systemd[1]: Started Ceph mgr.x for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T15:58:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:02 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/921675086' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-25T15:58:02.565 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:02.322+0000 7f29eb7a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T15:58:02.565 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:02.419+0000 7f29eb7a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T15:58:02.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-01-25T15:58:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-25T15:58:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-25T15:58:02.926 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:02.927 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-01-25T15:57:54.453924+0000", 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.281+0000 7f126da79700 1 Processor -- start 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.281+0000 7f126da79700 1 -- start start 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.281+0000 7f126da79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.281+0000 7f126da79700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1268106b20 con 0x7f1268106130 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:02.928 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57466/0 (socket says 172.21.15.33:57466) 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12677fe700 1 -- 172.21.15.33:0/2763994558 learned_addr learned my addr 172.21.15.33:0/2763994558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12677fe700 1 -- 172.21.15.33:0/2763994558 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1268107330 con 0x7f1268106130 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12677fe700 1 --2- 172.21.15.33:0/2763994558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f12500098a0 tx=0x7f1250009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9aefdb0d89d914 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12667fc700 1 -- 172.21.15.33:0/2763994558 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1250004160 con 0x7f1268106130 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12667fc700 1 -- 172.21.15.33:0/2763994558 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f12500042c0 con 0x7f1268106130 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.282+0000 7f12667fc700 1 -- 172.21.15.33:0/2763994558 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1250011a00 con 0x7f1268106130 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 -- 172.21.15.33:0/2763994558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 msgr2=0x7f1268106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 --2- 172.21.15.33:0/2763994558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f12500098a0 tx=0x7f1250009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 -- 172.21.15.33:0/2763994558 shutdown_connections 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 --2- 172.21.15.33:0/2763994558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f1268106550 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 -- 172.21.15.33:0/2763994558 >> 172.21.15.33:0/2763994558 conn(0x7f1268101710 msgr2=0x7f1268103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 -- 172.21.15.33:0/2763994558 shutdown_connections 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 -- 172.21.15.33:0/2763994558 wait complete. 2024-01-25T15:58:02.929 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.283+0000 7f126da79700 1 Processor -- start 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f126da79700 1 -- start start 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f126da79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f126da79700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126819b4c0 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f12677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f12677fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57482/0 (socket says 172.21.15.33:57482) 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f12677fe700 1 -- 172.21.15.33:0/921675086 learned_addr learned my addr 172.21.15.33:0/921675086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f12677fe700 1 -- 172.21.15.33:0/921675086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1250009550 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.284+0000 7f12677fe700 1 --2- 172.21.15.33:0/921675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1250009130 tx=0x7f1250011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12500038b0 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126819b6c0 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1250003a10 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126819bb60 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1250019550 con 0x7f1268106130 2024-01-25T15:58:02.930 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1250019c40 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.285+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1250024670 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.286+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1248005360 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.288+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f1250003b80 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.362+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f12480058e0 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.362+0000 7f1264ff9700 1 -- 172.21.15.33:0/921675086 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f125001a960 con 0x7f1268106130 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 msgr2=0x7f126819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 --2- 172.21.15.33:0/921675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1250009130 tx=0x7f1250011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 shutdown_connections 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 --2- 172.21.15.33:0/921675086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1268106130 0x7f126819af80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 >> 172.21.15.33:0/921675086 conn(0x7f1268101710 msgr2=0x7f1268102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 shutdown_connections 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:02.363+0000 7f126da79700 1 -- 172.21.15.33:0/921675086 wait complete. 2024-01-25T15:58:02.931 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-01-25T15:58:03.182 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.086+0000 7f29eb7a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T15:58:03.182 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.179+0000 7f29eb7a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T15:58:03.504 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.265+0000 7f29eb7a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T15:58:03.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.418+0000 7f29eb7a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T15:58:03.505 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.501+0000 7f29eb7a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T15:58:03.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.680+0000 7f29eb7a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T15:58:04.139 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.862+0000 7f29eb7a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:04.139 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:03.950+0000 7f29eb7a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T15:58:04.140 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:04.045+0000 7f29eb7a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T15:58:04.140 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:04.134+0000 7f29eb7a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T15:58:05.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:05.231+0000 7f29eb7a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T15:58:05.781 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:05.369+0000 7f29eb7a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T15:58:05.781 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:05.569+0000 7f29eb7a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T15:58:06.036 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1819522265' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-25T15:58:06.036 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:05.816+0000 7f29eb7a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T15:58:06.036 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:06.000+0000 7f29eb7a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-25T15:58:06.040 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-25T15:58:06.041 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:06.042 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-25T15:58:06.050 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-25T15:58:06.051 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-25T15:58:06.052 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.053 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-01-25T15:57:54.453924+0000", 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7cd635b700 1 Processor -- start 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7cd635b700 1 -- start start 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7cd635b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7cd635b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd007b8e0 con 0x7f7cd007cf10 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:06.054 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57498/0 (socket says 172.21.15.33:57498) 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7ccffff700 1 -- 172.21.15.33:0/1084106933 learned_addr learned my addr 172.21.15.33:0/1084106933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.336+0000 7f7ccffff700 1 -- 172.21.15.33:0/1084106933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd007ba20 con 0x7f7cd007cf10 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7ccffff700 1 --2- 172.21.15.33:0/1084106933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098a0 tx=0x7f7cc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f7e14067ec7be6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cceffd700 1 -- 172.21.15.33:0/1084106933 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0004160 con 0x7f7cd007cf10 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cceffd700 1 -- 172.21.15.33:0/1084106933 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cc00042c0 con 0x7f7cd007cf10 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cceffd700 1 -- 172.21.15.33:0/1084106933 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0011a00 con 0x7f7cd007cf10 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 -- 172.21.15.33:0/1084106933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 msgr2=0x7f7cd007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 --2- 172.21.15.33:0/1084106933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7cc00098a0 tx=0x7f7cc0009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 -- 172.21.15.33:0/1084106933 shutdown_connections 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 --2- 172.21.15.33:0/1084106933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd007cf10 0x7f7cd007b310 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:06.055 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 -- 172.21.15.33:0/1084106933 >> 172.21.15.33:0/1084106933 conn(0x7f7cd01028a0 msgr2=0x7f7cd0104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.337+0000 7f7cd635b700 1 -- 172.21.15.33:0/1084106933 shutdown_connections 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.338+0000 7f7cd635b700 1 -- 172.21.15.33:0/1084106933 wait complete. 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.338+0000 7f7cd635b700 1 Processor -- start 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.338+0000 7f7cd635b700 1 -- start start 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.338+0000 7f7cd635b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.338+0000 7f7cd635b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd019db00 con 0x7f7cd019d1a0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57512/0 (socket says 172.21.15.33:57512) 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccffff700 1 -- 172.21.15.33:0/1819522265 learned_addr learned my addr 172.21.15.33:0/1819522265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccffff700 1 -- 172.21.15.33:0/1819522265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc0009550 con 0x7f7cd019d1a0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccffff700 1 --2- 172.21.15.33:0/1819522265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009130 tx=0x7f7cc0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc0003b70 con 0x7f7cd019d1a0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7cc0003cd0 con 0x7f7cd019d1a0 2024-01-25T15:58:06.056 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd019dd00 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7cc00197d0 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd01a0870 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.339+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7cc00199c0 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.340+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cc000b760 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.340+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd007cf10 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.343+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7f7cc002a060 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.417+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7cd007d120 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.418+0000 7f7ccd7fa700 1 -- 172.21.15.33:0/1819522265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7cc0019e00 con 0x7f7cd019d1a0 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.418+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 msgr2=0x7f7cd019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.418+0000 7f7cd635b700 1 --2- 172.21.15.33:0/1819522265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009130 tx=0x7f7cc0011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.419+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 shutdown_connections 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.419+0000 7f7cd635b700 1 --2- 172.21.15.33:0/1819522265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd019d1a0 0x7f7cd019d5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:06.057 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.419+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 >> 172.21.15.33:0/1819522265 conn(0x7f7cd01028a0 msgr2=0x7f7cd01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:06.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.419+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 shutdown_connections 2024-01-25T15:58:06.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:05.419+0000 7f7cd635b700 1 -- 172.21.15.33:0/1819522265 wait complete. 2024-01-25T15:58:06.058 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-01-25T15:58:06.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:06.189+0000 7f29eb7a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T15:58:06.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:06.461+0000 7f29eb7a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:06.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:06.549+0000 7f29eb7a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:07.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:06.948+0000 7f29eb7a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T15:58:08.064 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:07.764+0000 7f29eb7a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T15:58:08.422 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:08.351+0000 7f29eb7a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T15:58:08.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:08 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/815854263' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-25T15:58:08.815 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:08.453+0000 7f29eb7a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T15:58:08.815 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:08.664+0000 7f29eb7a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T15:58:09.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:09.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:09.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T15:58:09.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-25T15:58:09.065 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-25T15:58:09.066 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-01-25T15:58:09.067 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-01-25T15:57:54.453924+0000", 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-25T15:58:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.425+0000 7fbfd8a99700 1 Processor -- start 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.425+0000 7fbfd8a99700 1 -- start start 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.425+0000 7fbfd8a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.425+0000 7fbfd8a99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd4107c20 con 0x7fbfd4107230 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57518/0 (socket says 172.21.15.33:57518) 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd259c700 1 -- 172.21.15.33:0/2883217969 learned_addr learned my addr 172.21.15.33:0/2883217969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd259c700 1 -- 172.21.15.33:0/2883217969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4108430 con 0x7fbfd4107230 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd259c700 1 --2- 172.21.15.33:0/2883217969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098a0 tx=0x7fbfc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1e618dc0b64ac8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd159a700 1 -- 172.21.15.33:0/2883217969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfc4004160 con 0x7fbfd4107230 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd159a700 1 -- 172.21.15.33:0/2883217969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfc40042c0 con 0x7fbfd4107230 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.426+0000 7fbfd159a700 1 -- 172.21.15.33:0/2883217969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfc4011a00 con 0x7fbfd4107230 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 -- 172.21.15.33:0/2883217969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 msgr2=0x7fbfd4107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 --2- 172.21.15.33:0/2883217969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40098a0 tx=0x7fbfc4009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 -- 172.21.15.33:0/2883217969 shutdown_connections 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 --2- 172.21.15.33:0/2883217969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd4107650 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 -- 172.21.15.33:0/2883217969 >> 172.21.15.33:0/2883217969 conn(0x7fbfd4102890 msgr2=0x7fbfd4104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 -- 172.21.15.33:0/2883217969 shutdown_connections 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.427+0000 7fbfd8a99700 1 -- 172.21.15.33:0/2883217969 wait complete. 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd8a99700 1 Processor -- start 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd8a99700 1 -- start start 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd8a99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd8a99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd419b4b0 con 0x7fbfd4107230 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57526/0 (socket says 172.21.15.33:57526) 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd259c700 1 -- 172.21.15.33:0/815854263 learned_addr learned my addr 172.21.15.33:0/815854263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd259c700 1 -- 172.21.15.33:0/815854263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc4009550 con 0x7fbfd4107230 2024-01-25T15:58:09.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.428+0000 7fbfd259c700 1 --2- 172.21.15.33:0/815854263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009130 tx=0x7fbfc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfc40038b0 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbfc4003a10 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd419b6b0 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfc4019550 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd419bb50 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbfc4019b90 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.429+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbfc4024600 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.430+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd4062e10 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.432+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73119 (secure 0 0 0) 0x7fbfc4003b80 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.506+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbfd419e6f0 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.506+0000 7fbfcb7fe700 1 -- 172.21.15.33:0/815854263 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fbfc401a960 con 0x7fbfd4107230 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 msgr2=0x7fbfd419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 --2- 172.21.15.33:0/815854263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009130 tx=0x7fbfc4011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:09.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 shutdown_connections 2024-01-25T15:58:09.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 --2- 172.21.15.33:0/815854263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfd4107230 0x7fbfd419af70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:09.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 >> 172.21.15.33:0/815854263 conn(0x7fbfd4102890 msgr2=0x7fbfd4103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:09.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 shutdown_connections 2024-01-25T15:58:09.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:08.507+0000 7fbfd8a99700 1 -- 172.21.15.33:0/815854263 wait complete. 2024-01-25T15:58:09.072 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (3/15)... 2024-01-25T15:58:09.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: Activating manager daemon x 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: mgrmap e2: x(active, starting, since 0.00294368s) 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: Manager daemon x is now available 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' 2024-01-25T15:58:09.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' 2024-01-25T15:58:09.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:09 smithi033 ceph-mon[31417]: from='mgr.14100 172.21.15.33:0/2741593141' entity='mgr.x' 2024-01-25T15:58:11.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:10 smithi033 ceph-mon[31417]: mgrmap e3: x(active, since 1.00692s) 2024-01-25T15:58:12.245 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-01-25T15:58:12.246 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-01-25T15:58:12.247 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-01-25T15:58:12.256 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-01-25T15:58:12.257 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-01-25T15:57:54.453924+0000", 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:12.258 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.414+0000 7f7964a16700 1 Processor -- start 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.414+0000 7f7964a16700 1 -- start start 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f7964a16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f7964a16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796007ade0 con 0x7f7960107d80 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f795e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f795e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48392/0 (socket says 172.21.15.33:48392) 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f795e59c700 1 -- 172.21.15.33:0/1184177546 learned_addr learned my addr 172.21.15.33:0/1184177546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.415+0000 7f795e59c700 1 -- 172.21.15.33:0/1184177546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796007af20 con 0x7f7960107d80 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f795e59c700 1 --2- 172.21.15.33:0/1184177546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79500098a0 tx=0x7f7950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1980a2a1326802ff server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f795d59a700 1 -- 172.21.15.33:0/1184177546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7950004160 con 0x7f7960107d80 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f795d59a700 1 -- 172.21.15.33:0/1184177546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f79500042c0 con 0x7f7960107d80 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f795d59a700 1 -- 172.21.15.33:0/1184177546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7950011a00 con 0x7f7960107d80 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f7964a16700 1 -- 172.21.15.33:0/1184177546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 msgr2=0x7f796010a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f7964a16700 1 --2- 172.21.15.33:0/1184177546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f79500098a0 tx=0x7f7950009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:12.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f7964a16700 1 -- 172.21.15.33:0/1184177546 shutdown_connections 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.416+0000 7f7964a16700 1 --2- 172.21.15.33:0/1184177546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796010a170 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 -- 172.21.15.33:0/1184177546 >> 172.21.15.33:0/1184177546 conn(0x7f7960101710 msgr2=0x7f7960103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 -- 172.21.15.33:0/1184177546 shutdown_connections 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 -- 172.21.15.33:0/1184177546 wait complete. 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 Processor -- start 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 -- start start 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.417+0000 7f7964a16700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796019b480 con 0x7f7960107d80 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f795e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f795e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48398/0 (socket says 172.21.15.33:48398) 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f795e59c700 1 -- 172.21.15.33:0/1964169754 learned_addr learned my addr 172.21.15.33:0/1964169754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f795e59c700 1 -- 172.21.15.33:0/1964169754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950009550 con 0x7f7960107d80 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f795e59c700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7950009130 tx=0x7f7950011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:12.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79500038b0 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7950003a10 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796019b680 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7950019550 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.418+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796019bb20 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.419+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 3) v1 ==== 45524+0+0 (secure 0 0 0) 0x7f7950019840 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.419+0000 7f794f7fe700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 0x7f794803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.419+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f795004c520 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.419+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796010b370 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.419+0000 7f795dd9b700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 0x7f794803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.420+0000 7f795dd9b700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 0x7f794803ad80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.429+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7950019af0 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.683+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7f7950019cd0 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.726+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7960105d90 con 0x7f7960107d80 2024-01-25T15:58:12.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.726+0000 7f794f7fe700 1 -- 172.21.15.33:0/1964169754 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7950019af0 con 0x7f7960107d80 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 msgr2=0x7f794803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 0x7f794803ad80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7954006fd0 tx=0x7f7954006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 msgr2=0x7f796019af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7950009130 tx=0x7f7950011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 shutdown_connections 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f79480388c0 0x7f794803ad80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 --2- 172.21.15.33:0/1964169754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7960107d80 0x7f796019af40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.728+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 >> 172.21.15.33:0/1964169754 conn(0x7f7960101710 msgr2=0x7f79601022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.729+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 shutdown_connections 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:11.729+0000 7f7964a16700 1 -- 172.21.15.33:0/1964169754 wait complete. 2024-01-25T15:58:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-01-25T15:58:12.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:12 smithi033 ceph-mon[31417]: mgrmap e4: x(active, since 3s) 2024-01-25T15:58:12.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1964169754' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-01-25T15:58:13.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.642+0000 7f398fdde700 1 Processor -- start 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.642+0000 7f398fdde700 1 -- start start 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.642+0000 7f398fdde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:13.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.642+0000 7f398fdde700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398807aec0 con 0x7f3988107cd0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398db7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398db7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48410/0 (socket says 172.21.15.33:48410) 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398db7a700 1 -- 172.21.15.33:0/884119215 learned_addr learned my addr 172.21.15.33:0/884119215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398db7a700 1 -- 172.21.15.33:0/884119215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398810a600 con 0x7f3988107cd0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398db7a700 1 --2- 172.21.15.33:0/884119215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39840098a0 tx=0x7f3984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bf0fd49655a32b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398cb78700 1 -- 172.21.15.33:0/884119215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3984004160 con 0x7f3988107cd0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398cb78700 1 -- 172.21.15.33:0/884119215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f39840042c0 con 0x7f3988107cd0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.643+0000 7f398cb78700 1 -- 172.21.15.33:0/884119215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3984011a00 con 0x7f3988107cd0 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- 172.21.15.33:0/884119215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 msgr2=0x7f398810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 --2- 172.21.15.33:0/884119215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f39840098a0 tx=0x7f3984009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- 172.21.15.33:0/884119215 shutdown_connections 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 --2- 172.21.15.33:0/884119215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f398810a0c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:13.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- 172.21.15.33:0/884119215 >> 172.21.15.33:0/884119215 conn(0x7f3988101740 msgr2=0x7f3988103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- 172.21.15.33:0/884119215 shutdown_connections 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- 172.21.15.33:0/884119215 wait complete. 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 Processor -- start 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.644+0000 7f398fdde700 1 -- start start 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398fdde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398fdde700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3988107a90 con 0x7f3988107cd0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398db7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398db7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48412/0 (socket says 172.21.15.33:48412) 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398db7a700 1 -- 172.21.15.33:0/2705452224 learned_addr learned my addr 172.21.15.33:0/2705452224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398db7a700 1 -- 172.21.15.33:0/2705452224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984009550 con 0x7f3988107cd0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398db7a700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3984009130 tx=0x7f3984011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3984003b70 con 0x7f3988107cd0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3984003cd0 con 0x7f3988107cd0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3988105c80 con 0x7f3988107cd0 2024-01-25T15:58:13.393 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39840197d0 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.645+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988106120 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.646+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7f3984019a10 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.646+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398810b390 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.646+0000 7f397effd700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 0x7f397403aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.646+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f398404cc50 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.646+0000 7f398d379700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 0x7f397403aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.647+0000 7f398d379700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 0x7f397403aeb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3978006fd0 tx=0x7f3978006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.651+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f398402d9a0 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.853+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f39881064c0 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.854+0000 7f397effd700 1 -- 172.21.15.33:0/2705452224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f398402d350 con 0x7f3988107cd0 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 msgr2=0x7f397403aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 0x7f397403aeb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3978006fd0 tx=0x7f3978006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 msgr2=0x7f3988107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3984009130 tx=0x7f3984011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 shutdown_connections 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f39740389f0 0x7f397403aeb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 --2- 172.21.15.33:0/2705452224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3988107cd0 0x7f3988107550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 >> 172.21.15.33:0/2705452224 conn(0x7f3988101740 msgr2=0x7f3988102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 shutdown_connections 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:12.856+0000 7f398fdde700 1 -- 172.21.15.33:0/2705452224 wait complete. 2024-01-25T15:58:13.395 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-01-25T15:58:13.792 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:13 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2705452224' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-01-25T15:58:15.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:14 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1501442316' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-01-25T15:58:15.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:14 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setuser ceph since I am not root 2024-01-25T15:58:15.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:14 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setgroup ceph since I am not root 2024-01-25T15:58:15.337 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:15 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:15.245+0000 7f3d01f6c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.804+0000 7f5cb2371700 1 Processor -- start 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.804+0000 7f5cb2371700 1 -- start start 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.804+0000 7f5cb2371700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.804+0000 7f5cb2371700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac07ade0 con 0x7f5cac107d90 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48428/0 (socket says 172.21.15.33:48428) 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5cabfff700 1 -- 172.21.15.33:0/2954835500 learned_addr learned my addr 172.21.15.33:0/2954835500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5cabfff700 1 -- 172.21.15.33:0/2954835500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac07af20 con 0x7f5cac107d90 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5cabfff700 1 --2- 172.21.15.33:0/2954835500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0098a0 tx=0x7f5c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52706ca55c45d0b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5caaffd700 1 -- 172.21.15.33:0/2954835500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c9c004160 con 0x7f5cac107d90 2024-01-25T15:58:15.345 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5caaffd700 1 -- 172.21.15.33:0/2954835500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c9c0042c0 con 0x7f5cac107d90 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.805+0000 7f5caaffd700 1 -- 172.21.15.33:0/2954835500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c9c011a00 con 0x7f5cac107d90 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- 172.21.15.33:0/2954835500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 msgr2=0x7f5cac10a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 --2- 172.21.15.33:0/2954835500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c0098a0 tx=0x7f5c9c009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- 172.21.15.33:0/2954835500 shutdown_connections 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 --2- 172.21.15.33:0/2954835500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac10a180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- 172.21.15.33:0/2954835500 >> 172.21.15.33:0/2954835500 conn(0x7f5cac101760 msgr2=0x7f5cac103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- 172.21.15.33:0/2954835500 shutdown_connections 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- 172.21.15.33:0/2954835500 wait complete. 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 Processor -- start 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.806+0000 7f5cb2371700 1 -- start start 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cb2371700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cb2371700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac19d630 con 0x7f5cac107d90 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48430/0 (socket says 172.21.15.33:48430) 2024-01-25T15:58:15.346 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cabfff700 1 -- 172.21.15.33:0/1501442316 learned_addr learned my addr 172.21.15.33:0/1501442316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cabfff700 1 -- 172.21.15.33:0/1501442316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c009550 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cabfff700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fb0 tx=0x7f5c9c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c9c003a40 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5c9c003ba0 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac19d830 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.807+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c9c01a750 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.808+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac19dcd0 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.808+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 45630+0+0 (secure 0 0 0) 0x7f5c9c01a9d0 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.808+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac10b310 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.809+0000 7f5ca97fa700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 0x7f5c9403ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.809+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c9c04c820 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.809+0000 7f5cab7fe700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 0x7f5c9403ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.809+0000 7f5cab7fe700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 0x7f5c9403ae60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0006fd0 tx=0x7f5ca0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:13.814+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5c9c018b80 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.069+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f5cac1a0610 con 0x7f5cac107d90 2024-01-25T15:58:15.347 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.687+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f5c9c018350 con 0x7f5cac107d90 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.688+0000 7f5ca97fa700 1 -- 172.21.15.33:0/1501442316 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f5c9c02d910 con 0x7f5cac107d90 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 msgr2=0x7f5c9403ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 0x7f5c9403ae60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0006fd0 tx=0x7f5ca0006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 msgr2=0x7f5cac19d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fb0 tx=0x7f5c9c011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 shutdown_connections 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f5c940389a0 0x7f5c9403ae60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 --2- 172.21.15.33:0/1501442316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5cac107d90 0x7f5cac19d0f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 >> 172.21.15.33:0/1501442316 conn(0x7f5cac101760 msgr2=0x7f5cac102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 shutdown_connections 2024-01-25T15:58:15.348 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:14.690+0000 7f5cb2371700 1 -- 172.21.15.33:0/1501442316 wait complete. 2024-01-25T15:58:15.648 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:15 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:15.340+0000 7f3d01f6c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T15:58:15.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:15 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1501442316' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-01-25T15:58:15.954 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:15 smithi033 ceph-mon[31417]: mgrmap e5: x(active, since 6s) 2024-01-25T15:58:16.209 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.021+0000 7f3d01f6c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T15:58:16.209 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.118+0000 7f3d01f6c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T15:58:16.209 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.205+0000 7f3d01f6c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T15:58:16.564 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.361+0000 7f3d01f6c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T15:58:16.564 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.446+0000 7f3d01f6c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:16.662 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96db51c700 1 Processor -- start 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96db51c700 1 -- start start 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96db51c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96db51c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d407ae60 con 0x7f96d4107d30 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96d92b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96d92b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48472/0 (socket says 172.21.15.33:48472) 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.745+0000 7f96d92b8700 1 -- 172.21.15.33:0/21852871 learned_addr learned my addr 172.21.15.33:0/21852871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.746+0000 7f96d92b8700 1 -- 172.21.15.33:0/21852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d410a660 con 0x7f96d4107d30 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.746+0000 7f96d92b8700 1 --2- 172.21.15.33:0/21852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098a0 tx=0x7f96d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f2ccd6b502ff75 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.746+0000 7f96cbfff700 1 -- 172.21.15.33:0/21852871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96d0004160 con 0x7f96d4107d30 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.746+0000 7f96cbfff700 1 -- 172.21.15.33:0/21852871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96d00042c0 con 0x7f96d4107d30 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.746+0000 7f96cbfff700 1 -- 172.21.15.33:0/21852871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96d0011a00 con 0x7f96d4107d30 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- 172.21.15.33:0/21852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 msgr2=0x7f96d410a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:16.663 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 --2- 172.21.15.33:0/21852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098a0 tx=0x7f96d0009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- 172.21.15.33:0/21852871 shutdown_connections 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 --2- 172.21.15.33:0/21852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d410a120 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- 172.21.15.33:0/21852871 >> 172.21.15.33:0/21852871 conn(0x7f96d4101740 msgr2=0x7f96d4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- 172.21.15.33:0/21852871 shutdown_connections 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- 172.21.15.33:0/21852871 wait complete. 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 Processor -- start 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.747+0000 7f96db51c700 1 -- start start 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96db51c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96db51c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4107aa0 con 0x7f96d4107d30 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96d92b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96d92b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48480/0 (socket says 172.21.15.33:48480) 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96d92b8700 1 -- 172.21.15.33:0/261200270 learned_addr learned my addr 172.21.15.33:0/261200270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96d92b8700 1 -- 172.21.15.33:0/261200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0009550 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96d92b8700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009130 tx=0x7f96d0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96d0003b70 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f96d0003cd0 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d4105ce0 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96d00197d0 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.748+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d4106180 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.749+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f96d001a430 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.749+0000 7f96ca7fc700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 0x7f96c003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.749+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f96d004cc90 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.749+0000 7f96d8ab7700 1 -- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 msgr2=0x7f96c003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.749+0000 7f96d8ab7700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 0x7f96c003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.750+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d4062e10 con 0x7f96d4107d30 2024-01-25T15:58:16.665 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.755+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f96d002d960 con 0x7f96d4107d30 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.950+0000 7f96d8ab7700 1 -- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 msgr2=0x7f96c003af00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:15.950+0000 7f96d8ab7700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 0x7f96c003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.035+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f96d4106550 con 0x7f96d4107d30 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.035+0000 7f96ca7fc700 1 -- 172.21.15.33:0/261200270 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f96d002d960 con 0x7f96d4107d30 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 msgr2=0x7f96c003af00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 0x7f96c003af00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 msgr2=0x7f96d4107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009130 tx=0x7f96d0011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 shutdown_connections 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7f96c0038a40 0x7f96c003af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.666 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 --2- 172.21.15.33:0/261200270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96d4107d30 0x7f96d4107560 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:16.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 >> 172.21.15.33:0/261200270 conn(0x7f96d4101740 msgr2=0x7f96d41021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:16.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 shutdown_connections 2024-01-25T15:58:16.667 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:16.037+0000 7f96db51c700 1 -- 172.21.15.33:0/261200270 wait complete. 2024-01-25T15:58:16.667 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-01-25T15:58:16.667 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-01-25T15:58:16.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/261200270' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-25T15:58:16.910 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.627+0000 7f3d01f6c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T15:58:16.910 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.815+0000 7f3d01f6c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:16.910 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:16.906+0000 7f3d01f6c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T15:58:17.243 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:17 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:17.005+0000 7f3d01f6c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T15:58:17.244 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:17 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:17.097+0000 7f3d01f6c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T15:58:18.446 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:18 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:18.168+0000 7f3d01f6c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T15:58:18.446 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:18 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:18.301+0000 7f3d01f6c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T15:58:18.731 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:18 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:18.499+0000 7f3d01f6c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T15:58:19.064 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:18 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:18.728+0000 7f3d01f6c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T15:58:19.064 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:18 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:18.912+0000 7f3d01f6c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:19.376 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:19.100+0000 7f3d01f6c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T15:58:19.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:19.373+0000 7f3d01f6c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:19.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:19.461+0000 7f3d01f6c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:20.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:19.858+0000 7f3d01f6c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T15:58:21.038 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:20 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:20.670+0000 7f3d01f6c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T15:58:21.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:21.252+0000 7f3d01f6c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T15:58:21.621 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:21.350+0000 7f3d01f6c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T15:58:21.621 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:21.554+0000 7f3d01f6c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T15:58:22.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: Active manager daemon x restarted 2024-01-25T15:58:22.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: Activating manager daemon x 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: osdmap e2: 0 total, 0 up, 0 in 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: mgrmap e6: x(active, starting, since 0.00361599s) 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: Manager daemon x is now available 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:22.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:21 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:22.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: Found migration_current of "None". Setting to last migration. 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:22.892 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:22 smithi033 ceph-mon[31417]: mgrmap e7: x(active, since 1.00566s) 2024-01-25T15:58:23.154 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed40e62700 1 Processor -- start 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed40e62700 1 -- start start 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed40e62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed40e62700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c107cb0 con 0x7fed3c1072c0 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48488/0 (socket says 172.21.15.33:48488) 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.036+0000 7fed3a59c700 1 -- 172.21.15.33:0/953345357 learned_addr learned my addr 172.21.15.33:0/953345357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.037+0000 7fed3a59c700 1 -- 172.21.15.33:0/953345357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c108470 con 0x7fed3c1072c0 2024-01-25T15:58:23.155 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.037+0000 7fed3a59c700 1 --2- 172.21.15.33:0/953345357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098a0 tx=0x7fed2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdde084380a363b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.037+0000 7fed3959a700 1 -- 172.21.15.33:0/953345357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed2c004160 con 0x7fed3c1072c0 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.037+0000 7fed3959a700 1 -- 172.21.15.33:0/953345357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed2c0042c0 con 0x7fed3c1072c0 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.037+0000 7fed3959a700 1 -- 172.21.15.33:0/953345357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed2c011a00 con 0x7fed3c1072c0 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- 172.21.15.33:0/953345357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 msgr2=0x7fed3c1076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 --2- 172.21.15.33:0/953345357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed2c0098a0 tx=0x7fed2c009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- 172.21.15.33:0/953345357 shutdown_connections 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 --2- 172.21.15.33:0/953345357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c1076e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- 172.21.15.33:0/953345357 >> 172.21.15.33:0/953345357 conn(0x7fed3c1028a0 msgr2=0x7fed3c104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- 172.21.15.33:0/953345357 shutdown_connections 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- 172.21.15.33:0/953345357 wait complete. 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 Processor -- start 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.038+0000 7fed40e62700 1 -- start start 2024-01-25T15:58:23.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed40e62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed40e62700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c07d0c0 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48492/0 (socket says 172.21.15.33:48492) 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed3a59c700 1 -- 172.21.15.33:0/274691796 learned_addr learned my addr 172.21.15.33:0/274691796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed3a59c700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c009550 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed3a59c700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed2c011710 tx=0x7fed2c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed2c003aa0 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fed2c003c00 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c07b2b0 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c07b750 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.039+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed2c024800 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fed2c024960 con 0x7fed3c1072c0 2024-01-25T15:58:23.157 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed2b7fe700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed1c000d40 con 0x7fed24038a90 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fed2c04d650 con 0x7fed3c1072c0 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed39d9b700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.040+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.241+0000 7fed39d9b700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.241+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.642+0000 7fed39d9b700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:17.642+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:18.443+0000 7fed39d9b700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:18.443+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:20.044+0000 7fed39d9b700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1043188550 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:20.044+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-25T15:58:23.158 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:21.569+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) v1 ==== 45408+0+0 (secure 0 0 0) 0x7fed2c02d9b0 con 0x7fed3c1072c0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:21.569+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 msgr2=0x7fed2403af50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:21.569+0000 7fed2b7fe700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.570+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) v1 ==== 45535+0+0 (secure 0 0 0) 0x7fed2c04e410 con 0x7fed3c1072c0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.570+0000 7fed2b7fe700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 0x7fed2403e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.570+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed1c000d40 con 0x7fed2403bff0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.571+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 0x7fed2403e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.571+0000 7fed39d9b700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 0x7fed2403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed30003a10 tx=0x7fed300092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.572+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fed1c000d40 con 0x7fed2403bff0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.578+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fed1c002800 con 0x7fed2403bff0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed2b7fe700 1 -- 172.21.15.33:0/274691796 <== mgr.14120 v2:172.21.15.33:6800/2443786608 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fed1c002800 con 0x7fed2403bff0 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 msgr2=0x7fed2403e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 0x7fed2403e3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fed30003a10 tx=0x7fed300092b0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 msgr2=0x7fed3c07cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed2c011710 tx=0x7fed2c011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.159 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 shutdown_connections 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fed2403bff0 0x7fed2403e3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed3c1072c0 0x7fed3c07cb80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 --2- 172.21.15.33:0/274691796 >> [v2:172.21.15.33:6800/1043188550,v1:172.21.15.33:6801/1043188550] conn(0x7fed24038a90 0x7fed2403af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 >> 172.21.15.33:0/274691796 conn(0x7fed3c1028a0 msgr2=0x7fed3c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 shutdown_connections 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:22.579+0000 7fed40e62700 1 -- 172.21.15.33:0/274691796 wait complete. 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-01-25T15:58:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-01-25T15:58:23.774 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:23 smithi033 ceph-mon[31417]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-25T15:58:23.774 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:23 smithi033 ceph-mon[31417]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-25T15:58:23.774 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:23 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:22] ENGINE Bus STARTING 2024-01-25T15:58:23.774 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:23 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:22] ENGINE Serving on https://172.21.15.33:7150 2024-01-25T15:58:23.774 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:23 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2ae768700 1 Processor -- start 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2ae768700 1 -- start start 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2ae768700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2ae768700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a807ae60 con 0x7fc2a8107cd0 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42832/0 (socket says 172.21.15.33:42832) 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.544+0000 7fc2a7fff700 1 -- 172.21.15.33:0/3122889244 learned_addr learned my addr 172.21.15.33:0/3122889244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2a7fff700 1 -- 172.21.15.33:0/3122889244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a810a600 con 0x7fc2a8107cd0 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2a7fff700 1 --2- 172.21.15.33:0/3122889244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2980098a0 tx=0x7fc298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70412b20cbea15ee server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:24.282 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2a6ffd700 1 -- 172.21.15.33:0/3122889244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc298004160 con 0x7fc2a8107cd0 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2a6ffd700 1 -- 172.21.15.33:0/3122889244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc2980042c0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2a6ffd700 1 -- 172.21.15.33:0/3122889244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc298011a00 con 0x7fc2a8107cd0 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2ae768700 1 -- 172.21.15.33:0/3122889244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 msgr2=0x7fc2a810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2ae768700 1 --2- 172.21.15.33:0/3122889244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2980098a0 tx=0x7fc298009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2ae768700 1 -- 172.21.15.33:0/3122889244 shutdown_connections 2024-01-25T15:58:24.283 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2ae768700 1 --2- 172.21.15.33:0/3122889244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a810a0c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.545+0000 7fc2ae768700 1 -- 172.21.15.33:0/3122889244 >> 172.21.15.33:0/3122889244 conn(0x7fc2a81016e0 msgr2=0x7fc2a8103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.546+0000 7fc2ae768700 1 -- 172.21.15.33:0/3122889244 shutdown_connections 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.546+0000 7fc2ae768700 1 -- 172.21.15.33:0/3122889244 wait complete. 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.546+0000 7fc2ae768700 1 Processor -- start 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.546+0000 7fc2ae768700 1 -- start start 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2ae768700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2ae768700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a819b410 con 0x7fc2a8107cd0 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42848/0 (socket says 172.21.15.33:42848) 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a7fff700 1 -- 172.21.15.33:0/2271917244 learned_addr learned my addr 172.21.15.33:0/2271917244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:24.284 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a7fff700 1 -- 172.21.15.33:0/2271917244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc298009550 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a7fff700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc298009130 tx=0x7fc298011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc298003b70 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc298003cd0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a819b610 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc2980197d0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.547+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a819bab0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.548+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) v1 ==== 45535+0+0 (secure 0 0 0) 0x7fc2980199c0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.548+0000 7fc2a57fa700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 0x7fc29003ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.548+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a810b330 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.548+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc29804cc50 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.549+0000 7fc2a77fe700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 0x7fc29003ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.549+0000 7fc2a77fe700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 0x7fc29003ae30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006fd0 tx=0x7fc29c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.553+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc2980188f0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.285 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.621+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fc298019cc0 con 0x7fc2a8107cd0 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.771+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2a8102930 con 0x7fc290038970 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.778+0000 7fc2a57fa700 1 -- 172.21.15.33:0/2271917244 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc2a8102930 con 0x7fc290038970 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.780+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 msgr2=0x7fc29003ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.780+0000 7fc2ae768700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 0x7fc29003ae30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006fd0 tx=0x7fc29c006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.780+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 msgr2=0x7fc2a819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.780+0000 7fc2ae768700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc298009130 tx=0x7fc298011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 shutdown_connections 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fc290038970 0x7fc29003ae30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 --2- 172.21.15.33:0/2271917244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a8107cd0 0x7fc2a819aed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 >> 172.21.15.33:0/2271917244 conn(0x7fc2a81016e0 msgr2=0x7fc2a8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 shutdown_connections 2024-01-25T15:58:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:23.781+0000 7fc2ae768700 1 -- 172.21.15.33:0/2271917244 wait complete. 2024-01-25T15:58:24.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:24 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:22] ENGINE Serving on http://172.21.15.33:8765 2024-01-25T15:58:24.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:24 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:22] ENGINE Bus STARTED 2024-01-25T15:58:24.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:24 smithi033 ceph-mon[31417]: mgrmap e8: x(active, since 2s) 2024-01-25T15:58:24.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:24 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:24.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:24 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.679+0000 7f8a591f8700 1 Processor -- start 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a591f8700 1 -- start start 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a591f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a591f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5407ae60 con 0x7f8a54107d00 2024-01-25T15:58:25.480 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42854/0 (socket says 172.21.15.33:42854) 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a52d9d700 1 -- 172.21.15.33:0/2494521308 learned_addr learned my addr 172.21.15.33:0/2494521308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a52d9d700 1 -- 172.21.15.33:0/2494521308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5410a630 con 0x7f8a54107d00 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a52d9d700 1 --2- 172.21.15.33:0/2494521308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009e80 tx=0x7f8a4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934499e22c9583ee server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.680+0000 7f8a51d9b700 1 -- 172.21.15.33:0/2494521308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a4400e070 con 0x7f8a54107d00 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a51d9b700 1 -- 172.21.15.33:0/2494521308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a4400b8c0 con 0x7f8a54107d00 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a51d9b700 1 -- 172.21.15.33:0/2494521308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a440044c0 con 0x7f8a54107d00 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 -- 172.21.15.33:0/2494521308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 msgr2=0x7f8a5410a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2494521308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a44009e80 tx=0x7f8a4400b330 comp rx=0 tx=0).stop 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 -- 172.21.15.33:0/2494521308 shutdown_connections 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2494521308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a5410a0f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 -- 172.21.15.33:0/2494521308 >> 172.21.15.33:0/2494521308 conn(0x7f8a54101710 msgr2=0x7f8a54103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 -- 172.21.15.33:0/2494521308 shutdown_connections 2024-01-25T15:58:25.481 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.681+0000 7f8a591f8700 1 -- 172.21.15.33:0/2494521308 wait complete. 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a591f8700 1 Processor -- start 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a591f8700 1 -- start start 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a591f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a591f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54107ac0 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a52d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42858/0 (socket says 172.21.15.33:42858) 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.682+0000 7f8a52d9d700 1 -- 172.21.15.33:0/2013142106 learned_addr learned my addr 172.21.15.33:0/2013142106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a52d9d700 1 -- 172.21.15.33:0/2013142106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44009b30 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a52d9d700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a440072a0 tx=0x7f8a440093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a4400e040 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8a4400bd70 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a54105cb0 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a4401c6e0 con 0x7f8a54107d00 2024-01-25T15:58:25.482 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.683+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a541061b0 con 0x7f8a54107d00 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.684+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f8a4401c8d0 con 0x7f8a54107d00 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.684+0000 7f8a4bfff700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 0x7f8a4003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.684+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5410b360 con 0x7f8a54107d00 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.684+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8a4404cab0 con 0x7f8a54107d00 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.684+0000 7f8a5259c700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 0x7f8a4003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.685+0000 7f8a5259c700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 0x7f8a4003af00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.689+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8a4401cdb0 con 0x7f8a54107d00 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.893+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a541068a0 con 0x7f8a40038a40 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.893+0000 7f8a4bfff700 1 -- 172.21.15.33:0/2013142106 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f8a541068a0 con 0x7f8a40038a40 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 msgr2=0x7f8a4003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 0x7f8a4003af00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006fd0 tx=0x7f8a3c006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 msgr2=0x7f8a54107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8a440072a0 tx=0x7f8a440093c0 comp rx=0 tx=0).stop 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 shutdown_connections 2024-01-25T15:58:25.483 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8a40038a40 0x7f8a4003af00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:25.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 --2- 172.21.15.33:0/2013142106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a54107d00 0x7f8a54107580 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:25.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 >> 172.21.15.33:0/2013142106 conn(0x7f8a54101710 msgr2=0x7f8a54102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:25.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.895+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 shutdown_connections 2024-01-25T15:58:25.484 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:24.896+0000 7f8a591f8700 1 -- 172.21.15.33:0/2013142106 wait complete. 2024-01-25T15:58:25.484 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-01-25T15:58:25.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:25 smithi033 ceph-mon[31417]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:26.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-mon[31417]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:26.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:26.960 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: Generating public/private rsa key pair. 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: Your identification has been saved in /tmp/tmp_hv9ewcn/key. 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: Your public key has been saved in /tmp/tmp_hv9ewcn/key.pub. 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: The key fingerprint is: 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: SHA256:iJ3/UaGm8o0ZvlPJIUWZoVVTBfoK98R9qfU+hcSG9eg ceph-55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: The key's randomart image is: 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: +---[RSA 3072]----+ 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | .+=o.oo.| 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | o+ o. | 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | .. o+ o | 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | o o. ...+=.o| 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | . + So+oo+++o| 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | . o++ +E +| 2024-01-25T15:58:26.961 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | . +.. ... o| 2024-01-25T15:58:26.962 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | +.* . o | 2024-01-25T15:58:26.962 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: | *oo o| 2024-01-25T15:58:26.962 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: +----[SHA256]-----+ 2024-01-25T15:58:27.236 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.902+0000 7fb60194a700 1 Processor -- start 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.902+0000 7fb60194a700 1 -- start start 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.902+0000 7fb60194a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.902+0000 7fb60194a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc07ae60 con 0x7fb5fc107d00 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.902+0000 7fb5faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42866/0 (socket says 172.21.15.33:42866) 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5faffd700 1 -- 172.21.15.33:0/285964287 learned_addr learned my addr 172.21.15.33:0/285964287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5faffd700 1 -- 172.21.15.33:0/285964287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc10a630 con 0x7fb5fc107d00 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5faffd700 1 --2- 172.21.15.33:0/285964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009e80 tx=0x7fb5f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c704e16d0726b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5f9ffb700 1 -- 172.21.15.33:0/285964287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f400e070 con 0x7fb5fc107d00 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5f9ffb700 1 -- 172.21.15.33:0/285964287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5f400b8c0 con 0x7fb5fc107d00 2024-01-25T15:58:27.237 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.903+0000 7fb5f9ffb700 1 -- 172.21.15.33:0/285964287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f40044c0 con 0x7fb5fc107d00 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 -- 172.21.15.33:0/285964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 msgr2=0x7fb5fc10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 --2- 172.21.15.33:0/285964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009e80 tx=0x7fb5f400b330 comp rx=0 tx=0).stop 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 -- 172.21.15.33:0/285964287 shutdown_connections 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 --2- 172.21.15.33:0/285964287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc10a0f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 -- 172.21.15.33:0/285964287 >> 172.21.15.33:0/285964287 conn(0x7fb5fc101710 msgr2=0x7fb5fc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 -- 172.21.15.33:0/285964287 shutdown_connections 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 -- 172.21.15.33:0/285964287 wait complete. 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.904+0000 7fb60194a700 1 Processor -- start 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb60194a700 1 -- start start 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb60194a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb60194a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5fc19b430 con 0x7fb5fc107d00 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb5faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb5faffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42876/0 (socket says 172.21.15.33:42876) 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb5faffd700 1 -- 172.21.15.33:0/1116035466 learned_addr learned my addr 172.21.15.33:0/1116035466 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:27.238 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb5faffd700 1 -- 172.21.15.33:0/1116035466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f4009b30 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.905+0000 7fb5faffd700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4004110 tx=0x7fb5f4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.906+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f400e040 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.906+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb5f400bd70 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.906+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc19b630 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.906+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5f401c670 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.906+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5fc19bb30 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fb5f4012430 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5fc10b360 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb600948700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 0x7fb5e803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb5f404ca00 con 0x7fb5fc107d00 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb5fa7fc700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 0x7fb5e803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:27.239 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.907+0000 7fb5fa7fc700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 0x7fb5e803af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:25.912+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb5f4011390 con 0x7fb5fc107d00 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.115+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5fc102930 con 0x7fb5e8038a40 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.612+0000 7fb600948700 1 -- 172.21.15.33:0/1116035466 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb5fc102930 con 0x7fb5e8038a40 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 msgr2=0x7fb5e803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 0x7fb5e803af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4006fd0 tx=0x7fb5e4006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 msgr2=0x7fb5fc19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4004110 tx=0x7fb5f4004610 comp rx=0 tx=0).stop 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 shutdown_connections 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fb5e8038a40 0x7fb5e803af00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 --2- 172.21.15.33:0/1116035466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5fc107d00 0x7fb5fc19aef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 >> 172.21.15.33:0/1116035466 conn(0x7fb5fc101710 msgr2=0x7fb5fc102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 shutdown_connections 2024-01-25T15:58:27.240 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:26.614+0000 7fb60194a700 1 -- 172.21.15.33:0/1116035466 wait complete. 2024-01-25T15:58:27.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:27 smithi033 ceph-mon[31417]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:27.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:27 smithi033 ceph-mon[31417]: Generating ssh key... 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f73011d3700 1 Processor -- start 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f73011d3700 1 -- start start 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f73011d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f73011d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc07ade0 con 0x7f72fc107d80 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f72fad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f72fad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42886/0 (socket says 172.21.15.33:42886) 2024-01-25T15:58:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.671+0000 7f72fad9d700 1 -- 172.21.15.33:0/1357452096 learned_addr learned my addr 172.21.15.33:0/1357452096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f72fad9d700 1 -- 172.21.15.33:0/1357452096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc07af20 con 0x7f72fc107d80 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f72fad9d700 1 --2- 172.21.15.33:0/1357452096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009e80 tx=0x7f72ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2bd7dad4254b1b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f72f9d9b700 1 -- 172.21.15.33:0/1357452096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ec00e070 con 0x7f72fc107d80 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f72f9d9b700 1 -- 172.21.15.33:0/1357452096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72ec00b8c0 con 0x7f72fc107d80 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f72f9d9b700 1 -- 172.21.15.33:0/1357452096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ec0044c0 con 0x7f72fc107d80 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f73011d3700 1 -- 172.21.15.33:0/1357452096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 msgr2=0x7f72fc10a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.672+0000 7f73011d3700 1 --2- 172.21.15.33:0/1357452096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009e80 tx=0x7f72ec00b330 comp rx=0 tx=0).stop 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 -- 172.21.15.33:0/1357452096 shutdown_connections 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 --2- 172.21.15.33:0/1357452096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc10a170 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 -- 172.21.15.33:0/1357452096 >> 172.21.15.33:0/1357452096 conn(0x7f72fc101710 msgr2=0x7f72fc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 -- 172.21.15.33:0/1357452096 shutdown_connections 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 -- 172.21.15.33:0/1357452096 wait complete. 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.673+0000 7f73011d3700 1 Processor -- start 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f73011d3700 1 -- start start 2024-01-25T15:58:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f73011d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f73011d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc107b40 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72fad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72fad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42900/0 (socket says 172.21.15.33:42900) 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72fad9d700 1 -- 172.21.15.33:0/947990632 learned_addr learned my addr 172.21.15.33:0/947990632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72fad9d700 1 -- 172.21.15.33:0/947990632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009b30 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72fad9d700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004110 tx=0x7f72ec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.674+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ec00e040 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.675+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f72ec00bd70 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.675+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc105d30 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.675+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72ec01c670 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.675+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc106230 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.675+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f72ec01c8b0 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.676+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc062e10 con 0x7f72fc107d80 2024-01-25T15:58:28.312 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.676+0000 7f72f3fff700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 0x7f72e803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.676+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f72ec04cc20 con 0x7f72fc107d80 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.676+0000 7f72fa59c700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 0x7f72e803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.676+0000 7f72fa59c700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 0x7f72e803af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006fd0 tx=0x7f72e4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.681+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f72ec011980 con 0x7f72fc107d80 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.884+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f72fc106860 con 0x7f72e8038a40 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.885+0000 7f72f3fff700 1 -- 172.21.15.33:0/947990632 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f72fc106860 con 0x7f72e8038a40 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.886+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 msgr2=0x7f72e803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.886+0000 7f73011d3700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 0x7f72e803af00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72e4006fd0 tx=0x7f72e4006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.886+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 msgr2=0x7f72fc107600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.886+0000 7f73011d3700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72ec004110 tx=0x7f72ec004610 comp rx=0 tx=0).stop 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 shutdown_connections 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f72e8038a40 0x7f72e803af00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 --2- 172.21.15.33:0/947990632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72fc107d80 0x7f72fc107600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 >> 172.21.15.33:0/947990632 conn(0x7f72fc101710 msgr2=0x7f72fc102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:28.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 shutdown_connections 2024-01-25T15:58:28.314 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:27.887+0000 7f73011d3700 1 -- 172.21.15.33:0/947990632 wait complete. 2024-01-25T15:58:28.314 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-01-25T15:58:28.314 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-01-25T15:58:28.314 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-01-25T15:58:28.973 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:28 smithi033 ceph-mon[31417]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:29.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:29 smithi033 ceph-mon[31417]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:29.941 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:29 smithi033 ceph-mon[31417]: Deploying cephadm binary to smithi033 2024-01-25T15:58:33.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-01-25T15:58:33.795 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd2551e6700 1 Processor -- start 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd2551e6700 1 -- start start 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd2551e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd2551e6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25007ae60 con 0x7fd250107cd0 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51772/0 (socket says 172.21.15.33:51772) 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd24ed9d700 1 -- 172.21.15.33:0/3207845112 learned_addr learned my addr 172.21.15.33:0/3207845112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:33.796 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.757+0000 7fd24ed9d700 1 -- 172.21.15.33:0/3207845112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25010a600 con 0x7fd250107cd0 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd24ed9d700 1 --2- 172.21.15.33:0/3207845112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c1a50a616f7319 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd24dd9b700 1 -- 172.21.15.33:0/3207845112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240004160 con 0x7fd250107cd0 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd24dd9b700 1 -- 172.21.15.33:0/3207845112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd2400042c0 con 0x7fd250107cd0 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd24dd9b700 1 -- 172.21.15.33:0/3207845112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240011a00 con 0x7fd250107cd0 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 -- 172.21.15.33:0/3207845112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 msgr2=0x7fd25010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 --2- 172.21.15.33:0/3207845112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd2400098a0 tx=0x7fd240009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 -- 172.21.15.33:0/3207845112 shutdown_connections 2024-01-25T15:58:33.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 --2- 172.21.15.33:0/3207845112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25010a0c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 -- 172.21.15.33:0/3207845112 >> 172.21.15.33:0/3207845112 conn(0x7fd2501016e0 msgr2=0x7fd250103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.758+0000 7fd2551e6700 1 -- 172.21.15.33:0/3207845112 shutdown_connections 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.759+0000 7fd2551e6700 1 -- 172.21.15.33:0/3207845112 wait complete. 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.759+0000 7fd2551e6700 1 Processor -- start 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.759+0000 7fd2551e6700 1 -- start start 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd2551e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd2551e6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25019b410 con 0x7fd250107cd0 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd24ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51786/0 (socket says 172.21.15.33:51786) 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd24ed9d700 1 -- 172.21.15.33:0/2395056725 learned_addr learned my addr 172.21.15.33:0/2395056725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd24ed9d700 1 -- 172.21.15.33:0/2395056725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd240009550 con 0x7fd250107cd0 2024-01-25T15:58:33.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd24ed9d700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd240009130 tx=0x7fd240011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd240003b70 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd240003cd0 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25019b610 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2400197d0 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.760+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25019bab0 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.761+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7fd240019a10 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.761+0000 7fd237fff700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 0x7fd23803af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.761+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25010b330 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.762+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd24004ccd0 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.762+0000 7fd24e59c700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 0x7fd23803af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.762+0000 7fd24e59c700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 0x7fd23803af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd244006fd0 tx=0x7fd244006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.766+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd240018db0 con 0x7fd250107cd0 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:28.970+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) v1 -- 0x7fd250102930 con 0x7fd238038a40 2024-01-25T15:58:33.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.224+0000 7fd237fff700 1 -- 172.21.15.33:0/2395056725 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd250102930 con 0x7fd238038a40 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 msgr2=0x7fd23803af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 0x7fd23803af00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd244006fd0 tx=0x7fd244006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 msgr2=0x7fd25019aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd240009130 tx=0x7fd240011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 shutdown_connections 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7fd238038a40 0x7fd23803af00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 --2- 172.21.15.33:0/2395056725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd250107cd0 0x7fd25019aed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 >> 172.21.15.33:0/2395056725 conn(0x7fd2501016e0 msgr2=0x7fd250102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 shutdown_connections 2024-01-25T15:58:33.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:33.226+0000 7fd2551e6700 1 -- 172.21.15.33:0/2395056725 wait complete. 2024-01-25T15:58:33.801 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-01-25T15:58:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:34 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:34 smithi033 ceph-mon[31417]: Added host smithi033 2024-01-25T15:58:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:34 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:35.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-01-25T15:58:35.167 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.239+0000 7f82b082f700 1 Processor -- start 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.239+0000 7f82b082f700 1 -- start start 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.239+0000 7f82b082f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.239+0000 7f82b082f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac07b850 con 0x7f82ac07ce10 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51794/0 (socket says 172.21.15.33:51794) 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82aa59c700 1 -- 172.21.15.33:0/2374332072 learned_addr learned my addr 172.21.15.33:0/2374332072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82aa59c700 1 -- 172.21.15.33:0/2374332072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac07b990 con 0x7f82ac07ce10 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82aa59c700 1 --2- 172.21.15.33:0/2374332072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f829c0098a0 tx=0x7f829c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72a760f2cb3dea6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82a959a700 1 -- 172.21.15.33:0/2374332072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f829c004160 con 0x7f82ac07ce10 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82a959a700 1 -- 172.21.15.33:0/2374332072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f829c0042c0 con 0x7f82ac07ce10 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.240+0000 7f82a959a700 1 -- 172.21.15.33:0/2374332072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f829c011a00 con 0x7f82ac07ce10 2024-01-25T15:58:35.168 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 -- 172.21.15.33:0/2374332072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 msgr2=0x7f82ac07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 --2- 172.21.15.33:0/2374332072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f829c0098a0 tx=0x7f829c009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 -- 172.21.15.33:0/2374332072 shutdown_connections 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 --2- 172.21.15.33:0/2374332072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac07b310 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 -- 172.21.15.33:0/2374332072 >> 172.21.15.33:0/2374332072 conn(0x7f82ac102830 msgr2=0x7f82ac104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 -- 172.21.15.33:0/2374332072 shutdown_connections 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.241+0000 7f82b082f700 1 -- 172.21.15.33:0/2374332072 wait complete. 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82b082f700 1 Processor -- start 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82b082f700 1 -- start start 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82b082f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:35.169 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82b082f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac19d640 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82aa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51806/0 (socket says 172.21.15.33:51806) 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82aa59c700 1 -- 172.21.15.33:0/2088048984 learned_addr learned my addr 172.21.15.33:0/2088048984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.242+0000 7f82aa59c700 1 -- 172.21.15.33:0/2088048984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c009550 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f82aa59c700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f829c009fb0 tx=0x7f829c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f829c003a40 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f829c003ba0 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac19d840 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f829c01a750 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.243+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac19dce0 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f829c01a980 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f829b7fe700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 0x7f829403af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac10d520 con 0x7f82ac07ce10 2024-01-25T15:58:35.170 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f829c04cbe0 con 0x7f82ac07ce10 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f82a9d9b700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 0x7f829403af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.244+0000 7f82a9d9b700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 0x7f829403af00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82a0006fd0 tx=0x7f82a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.249+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f829c018db0 con 0x7f82ac07ce10 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.476+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f82ac103af0 con 0x7f8294038a40 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.483+0000 7f829b7fe700 1 -- 172.21.15.33:0/2088048984 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f82ac103af0 con 0x7f8294038a40 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 msgr2=0x7f829403af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 0x7f829403af00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f82a0006fd0 tx=0x7f82a0006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 msgr2=0x7f82ac19d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:35.171 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f829c009fb0 tx=0x7f829c011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 shutdown_connections 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8294038a40 0x7f829403af00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 --2- 172.21.15.33:0/2088048984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82ac07ce10 0x7f82ac19d100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.485+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 >> 172.21.15.33:0/2088048984 conn(0x7f82ac102830 msgr2=0x7f82ac1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.486+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 shutdown_connections 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:34.486+0000 7f82b082f700 1 -- 172.21.15.33:0/2088048984 wait complete. 2024-01-25T15:58:35.172 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-01-25T15:58:35.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:35 smithi033 ceph-mon[31417]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:35.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:35 smithi033 ceph-mon[31417]: Saving service mon spec with placement count:5 2024-01-25T15:58:35.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:35 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:37.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-01-25T15:58:37.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f738c0bc700 1 Processor -- start 2024-01-25T15:58:37.071 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f738c0bc700 1 -- start start 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f738c0bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f738c0bc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7384107c70 con 0x7f7384107280 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f7389e58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f7389e58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51820/0 (socket says 172.21.15.33:51820) 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.584+0000 7f7389e58700 1 -- 172.21.15.33:0/2918745743 learned_addr learned my addr 172.21.15.33:0/2918745743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.585+0000 7f7389e58700 1 -- 172.21.15.33:0/2918745743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73841083e0 con 0x7f7384107280 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.585+0000 7f7389e58700 1 --2- 172.21.15.33:0/2918745743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73800098a0 tx=0x7f7380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da778152513ac2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.585+0000 7f7388e56700 1 -- 172.21.15.33:0/2918745743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7380004160 con 0x7f7384107280 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.585+0000 7f7388e56700 1 -- 172.21.15.33:0/2918745743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f73800042c0 con 0x7f7384107280 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.585+0000 7f7388e56700 1 -- 172.21.15.33:0/2918745743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7380011a00 con 0x7f7384107280 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- 172.21.15.33:0/2918745743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 msgr2=0x7f73841076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2918745743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73800098a0 tx=0x7f7380009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- 172.21.15.33:0/2918745743 shutdown_connections 2024-01-25T15:58:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2918745743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f73841076a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- 172.21.15.33:0/2918745743 >> 172.21.15.33:0/2918745743 conn(0x7f7384102880 msgr2=0x7f7384104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- 172.21.15.33:0/2918745743 shutdown_connections 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- 172.21.15.33:0/2918745743 wait complete. 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 Processor -- start 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- start start 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.586+0000 7f738c0bc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738419b480 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f7389e58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f7389e58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51830/0 (socket says 172.21.15.33:51830) 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f7389e58700 1 -- 172.21.15.33:0/2648358666 learned_addr learned my addr 172.21.15.33:0/2648358666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f7389e58700 1 -- 172.21.15.33:0/2648358666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7380009550 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f7389e58700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7380009fb0 tx=0x7f7380011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7380003a40 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7380003ba0 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738419b680 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f738001a750 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.587+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738419bb20 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.588+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f738001a9d0 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.588+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738410b610 con 0x7f7384107280 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.588+0000 7f737affd700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 0x7f737003aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.588+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f738004cb50 con 0x7f7384107280 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.588+0000 7f7389657700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 0x7f737003aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.589+0000 7f7389657700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 0x7f737003aeb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7374006fd0 tx=0x7f7374006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.593+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7380018350 con 0x7f7384107280 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.812+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7384103b20 con 0x7f73700389f0 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.819+0000 7f737affd700 1 -- 172.21.15.33:0/2648358666 <== mgr.14120 v2:172.21.15.33:6800/2443786608 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7384103b20 con 0x7f73700389f0 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 msgr2=0x7f737003aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 0x7f737003aeb0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7374006fd0 tx=0x7f7374006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 msgr2=0x7f738419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7380009fb0 tx=0x7f7380011ad0 comp rx=0 tx=0).stop 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 shutdown_connections 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f73700389f0 0x7f737003aeb0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 --2- 172.21.15.33:0/2648358666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7384107280 0x7f738419af40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.821+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 >> 172.21.15.33:0/2648358666 conn(0x7f7384102880 msgr2=0x7f7384103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:37.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.822+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 shutdown_connections 2024-01-25T15:58:37.077 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:35.822+0000 7f738c0bc700 1 -- 172.21.15.33:0/2648358666 wait complete. 2024-01-25T15:58:37.190 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:36 smithi033 ceph-mon[31417]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:36 smithi033 ceph-mon[31417]: Saving service mgr spec with placement count:2 2024-01-25T15:58:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:36 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:37.191 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:36 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:38.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:38 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:38.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:38 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1339695563' entity='client.admin' 2024-01-25T15:58:38.839 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3d0622700 1 Processor -- start 2024-01-25T15:58:38.839 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3d0622700 1 -- start start 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3d0622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3d0622700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c8107cb0 con 0x7ff3c81072c0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3ce3be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3ce3be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51838/0 (socket says 172.21.15.33:51838) 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.544+0000 7ff3ce3be700 1 -- 172.21.15.33:0/303135275 learned_addr learned my addr 172.21.15.33:0/303135275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.545+0000 7ff3ce3be700 1 -- 172.21.15.33:0/303135275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c8108470 con 0x7ff3c81072c0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.545+0000 7ff3ce3be700 1 --2- 172.21.15.33:0/303135275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40098a0 tx=0x7ff3c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11de8ee50d74d12d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.545+0000 7ff3cd3bc700 1 -- 172.21.15.33:0/303135275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4004160 con 0x7ff3c81072c0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.545+0000 7ff3cd3bc700 1 -- 172.21.15.33:0/303135275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c40042c0 con 0x7ff3c81072c0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.545+0000 7ff3cd3bc700 1 -- 172.21.15.33:0/303135275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3c40119b0 con 0x7ff3c81072c0 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- 172.21.15.33:0/303135275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 msgr2=0x7ff3c81076e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 --2- 172.21.15.33:0/303135275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40098a0 tx=0x7ff3c4009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:38.840 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- 172.21.15.33:0/303135275 shutdown_connections 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 --2- 172.21.15.33:0/303135275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c81076e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- 172.21.15.33:0/303135275 >> 172.21.15.33:0/303135275 conn(0x7ff3c81028a0 msgr2=0x7ff3c8104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- 172.21.15.33:0/303135275 shutdown_connections 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- 172.21.15.33:0/303135275 wait complete. 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 Processor -- start 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.546+0000 7ff3d0622700 1 -- start start 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3d0622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3d0622700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c819b520 con 0x7ff3c81072c0 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3ce3be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3ce3be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51850/0 (socket says 172.21.15.33:51850) 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3ce3be700 1 -- 172.21.15.33:0/1339695563 learned_addr learned my addr 172.21.15.33:0/1339695563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3ce3be700 1 -- 172.21.15.33:0/1339695563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4009550 con 0x7ff3c81072c0 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3ce3be700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4011670 tx=0x7ff3c4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4003af0 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c819b720 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff3c4003c50 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3c4024820 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.547+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c819bbc0 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.548+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7ff3c4024aa0 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.548+0000 7ff3bf7fe700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 0x7ff3b403af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.548+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff3c404ce30 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.548+0000 7ff3cdbbd700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 0x7ff3b403af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.549+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c810b6a0 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.549+0000 7ff3cdbbd700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 0x7ff3b403af00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8006fd0 tx=0x7ff3b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.558+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff3c401a430 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.779+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff3c819e5d0 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.781+0000 7ff3bf7fe700 1 -- 172.21.15.33:0/1339695563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff3c4018b00 con 0x7ff3c81072c0 2024-01-25T15:58:38.842 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 msgr2=0x7ff3b403af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 0x7ff3b403af00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8006fd0 tx=0x7ff3b8006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 msgr2=0x7ff3c819afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4011670 tx=0x7ff3c4011a30 comp rx=0 tx=0).stop 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 shutdown_connections 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7ff3b4038a40 0x7ff3b403af00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 --2- 172.21.15.33:0/1339695563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c81072c0 0x7ff3c819afe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 >> 172.21.15.33:0/1339695563 conn(0x7ff3c81028a0 msgr2=0x7ff3c81033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 shutdown_connections 2024-01-25T15:58:38.843 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:37.784+0000 7ff3d0622700 1 -- 172.21.15.33:0/1339695563 wait complete. 2024-01-25T15:58:40.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:39 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:40.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:39 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1577749053' entity='client.admin' 2024-01-25T15:58:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20fe26e700 1 Processor -- start 2024-01-25T15:58:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20fe26e700 1 -- start start 2024-01-25T15:58:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20fe26e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20fe26e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f807ae80 con 0x7f20f8107cf0 2024-01-25T15:58:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39998/0 (socket says 172.21.15.33:39998) 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20f7fff700 1 -- 172.21.15.33:0/145025796 learned_addr learned my addr 172.21.15.33:0/145025796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.261+0000 7f20f7fff700 1 -- 172.21.15.33:0/145025796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f810a620 con 0x7f20f8107cf0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20f7fff700 1 --2- 172.21.15.33:0/145025796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098a0 tx=0x7f20e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3435ba9e99490 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20f6ffd700 1 -- 172.21.15.33:0/145025796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e8004160 con 0x7f20f8107cf0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20f6ffd700 1 -- 172.21.15.33:0/145025796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e80042c0 con 0x7f20f8107cf0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20f6ffd700 1 -- 172.21.15.33:0/145025796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e80119b0 con 0x7f20f8107cf0 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20fe26e700 1 -- 172.21.15.33:0/145025796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 msgr2=0x7f20f810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.262+0000 7f20fe26e700 1 --2- 172.21.15.33:0/145025796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098a0 tx=0x7f20e8009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 -- 172.21.15.33:0/145025796 shutdown_connections 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 --2- 172.21.15.33:0/145025796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f810a0e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 -- 172.21.15.33:0/145025796 >> 172.21.15.33:0/145025796 conn(0x7f20f8101760 msgr2=0x7f20f8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 -- 172.21.15.33:0/145025796 shutdown_connections 2024-01-25T15:58:40.059 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 -- 172.21.15.33:0/145025796 wait complete. 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 Processor -- start 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.263+0000 7f20fe26e700 1 -- start start 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20fe26e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20fe26e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8107ab0 con 0x7f20f8107cf0 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40010/0 (socket says 172.21.15.33:40010) 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f7fff700 1 -- 172.21.15.33:0/1577749053 learned_addr learned my addr 172.21.15.33:0/1577749053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f7fff700 1 -- 172.21.15.33:0/1577749053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e8009550 con 0x7f20f8107cf0 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f7fff700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006af0 tx=0x7f20e8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e8003a40 con 0x7f20f8107cf0 2024-01-25T15:58:40.060 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e8003ba0 con 0x7f20f8107cf0 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f8105ca0 con 0x7f20f8107cf0 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20e80196c0 con 0x7f20f8107cf0 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.264+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8106140 con 0x7f20f8107cf0 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.265+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7f20e8019980 con 0x7f20f8107cf0 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.265+0000 7f20f57fa700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 0x7f20e003aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:40.061 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.265+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f810b400 con 0x7f20f8107cf0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.265+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f20e804ccf0 con 0x7f20f8107cf0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.266+0000 7f20f77fe700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 0x7f20e003aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.266+0000 7f20f77fe700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 0x7f20e003aeb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006fd0 tx=0x7f20ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.270+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f20e80243f0 con 0x7f20f8107cf0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.477+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f20f810b610 con 0x7f20f8107cf0 2024-01-25T15:58:40.062 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.479+0000 7f20f57fa700 1 -- 172.21.15.33:0/1577749053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f20e8023390 con 0x7f20f8107cf0 2024-01-25T15:58:40.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 msgr2=0x7f20e003aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:40.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 0x7f20e003aeb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f20ec006fd0 tx=0x7f20ec006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:40.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 msgr2=0x7f20f8107570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:40.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006af0 tx=0x7f20e8011a10 comp rx=0 tx=0).stop 2024-01-25T15:58:40.063 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 shutdown_connections 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f20e00389f0 0x7f20e003aeb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 --2- 172.21.15.33:0/1577749053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20f8107cf0 0x7f20f8107570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 >> 172.21.15.33:0/1577749053 conn(0x7f20f8101760 msgr2=0x7f20f81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.481+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 shutdown_connections 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:39.482+0000 7f20fe26e700 1 -- 172.21.15.33:0/1577749053 wait complete. 2024-01-25T15:58:40.064 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-01-25T15:58:41.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-mon[31417]: from='mgr.14120 172.21.15.33:0/3864362858' entity='mgr.x' 2024-01-25T15:58:41.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2350695386' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-01-25T15:58:41.565 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setuser ceph since I am not root 2024-01-25T15:58:41.565 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setgroup ceph since I am not root 2024-01-25T15:58:41.874 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:41.688+0000 7f95d561b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T15:58:41.875 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:41.782+0000 7f95d561b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T15:58:41.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.511+0000 7efda15d8700 1 Processor -- start 2024-01-25T15:58:41.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efda15d8700 1 -- start start 2024-01-25T15:58:41.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efda15d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:41.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efda15d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c105eb0 con 0x7efd9c1054c0 2024-01-25T15:58:41.886 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efd9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efd9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40024/0 (socket says 172.21.15.33:40024) 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.512+0000 7efd9affd700 1 -- 172.21.15.33:0/1046035531 learned_addr learned my addr 172.21.15.33:0/1046035531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.513+0000 7efd9affd700 1 -- 172.21.15.33:0/1046035531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c105ff0 con 0x7efd9c1054c0 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.513+0000 7efd9affd700 1 --2- 172.21.15.33:0/1046035531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098a0 tx=0x7efd8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca3579c7d0d19d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.513+0000 7efd99ffb700 1 -- 172.21.15.33:0/1046035531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8c004160 con 0x7efd9c1054c0 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.513+0000 7efd99ffb700 1 -- 172.21.15.33:0/1046035531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c0042c0 con 0x7efd9c1054c0 2024-01-25T15:58:41.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.513+0000 7efd99ffb700 1 -- 172.21.15.33:0/1046035531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8c0119b0 con 0x7efd9c1054c0 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 -- 172.21.15.33:0/1046035531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 msgr2=0x7efd9c1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 --2- 172.21.15.33:0/1046035531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0098a0 tx=0x7efd8c009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 -- 172.21.15.33:0/1046035531 shutdown_connections 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 --2- 172.21.15.33:0/1046035531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c1058e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 -- 172.21.15.33:0/1046035531 >> 172.21.15.33:0/1046035531 conn(0x7efd9c100ee0 msgr2=0x7efd9c103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 -- 172.21.15.33:0/1046035531 shutdown_connections 2024-01-25T15:58:41.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.514+0000 7efda15d8700 1 -- 172.21.15.33:0/1046035531 wait complete. 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efda15d8700 1 Processor -- start 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efda15d8700 1 -- start start 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efda15d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efda15d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9c108260 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efd9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efd9affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40030/0 (socket says 172.21.15.33:40030) 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.515+0000 7efd9affd700 1 -- 172.21.15.33:0/2350695386 learned_addr learned my addr 172.21.15.33:0/2350695386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efd9affd700 1 -- 172.21.15.33:0/2350695386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd8c009550 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efd9affd700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd8c003e60 tx=0x7efd8c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8c003a60 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd9c106450 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd8c003bc0 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd8c0196c0 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.516+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9c1068f0 con 0x7efd9c1054c0 2024-01-25T15:58:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.517+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9c062e10 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.518+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 45641+0+0 (secure 0 0 0) 0x7efd8c01ab60 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.518+0000 7efd83fff700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 0x7efd8403af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.518+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd8c04cc80 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.518+0000 7efd9a7fc700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 0x7efd8403af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.519+0000 7efd9a7fc700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 0x7efd8403af00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.526+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7efd8c00bbd0 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:40.806+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7efd9c106b30 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.129+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7efd8c02c3a0 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.130+0000 7efd83fff700 1 -- 172.21.15.33:0/2350695386 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7efd8c04b730 con 0x7efd9c1054c0 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 msgr2=0x7efd8403af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 0x7efd8403af00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efd90006fd0 tx=0x7efd90006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 msgr2=0x7efd9c107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:41.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efd8c003e60 tx=0x7efd8c011a30 comp rx=0 tx=0).stop 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 shutdown_connections 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7efd84038a40 0x7efd8403af00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 --2- 172.21.15.33:0/2350695386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd9c1054c0 0x7efd9c107d20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.132+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 >> 172.21.15.33:0/2350695386 conn(0x7efd9c100ee0 msgr2=0x7efd9c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.133+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 shutdown_connections 2024-01-25T15:58:41.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:41.133+0000 7efda15d8700 1 -- 172.21.15.33:0/2350695386 wait complete. 2024-01-25T15:58:42.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2350695386' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-01-25T15:58:42.447 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-mon[31417]: mgrmap e9: x(active, since 19s) 2024-01-25T15:58:42.722 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:42.444+0000 7f95d561b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T15:58:42.722 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:42.538+0000 7f95d561b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T15:58:42.722 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:42.629+0000 7f95d561b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T15:58:43.020 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:42.785+0000 7f95d561b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T15:58:43.020 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:42.870+0000 7f95d561b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e76528700 1 Processor -- start 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e76528700 1 -- start start 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e76528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e76528700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7007aec0 con 0x7f8e70105a90 2024-01-25T15:58:43.259 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40066/0 (socket says 172.21.15.33:40066) 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.318+0000 7f8e6ffff700 1 -- 172.21.15.33:0/2199863525 learned_addr learned my addr 172.21.15.33:0/2199863525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.319+0000 7f8e6ffff700 1 -- 172.21.15.33:0/2199863525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e701063f0 con 0x7f8e70105a90 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.319+0000 7f8e6ffff700 1 --2- 172.21.15.33:0/2199863525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8e600098a0 tx=0x7f8e60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6ce114e4b461c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.319+0000 7f8e6effd700 1 -- 172.21.15.33:0/2199863525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e60004160 con 0x7f8e70105a90 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.319+0000 7f8e6effd700 1 -- 172.21.15.33:0/2199863525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e600042c0 con 0x7f8e70105a90 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.319+0000 7f8e6effd700 1 -- 172.21.15.33:0/2199863525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e600119b0 con 0x7f8e70105a90 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- 172.21.15.33:0/2199863525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 msgr2=0x7f8e70105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 --2- 172.21.15.33:0/2199863525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8e600098a0 tx=0x7f8e60009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- 172.21.15.33:0/2199863525 shutdown_connections 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 --2- 172.21.15.33:0/2199863525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e70105eb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- 172.21.15.33:0/2199863525 >> 172.21.15.33:0/2199863525 conn(0x7f8e70101710 msgr2=0x7f8e70103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- 172.21.15.33:0/2199863525 shutdown_connections 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- 172.21.15.33:0/2199863525 wait complete. 2024-01-25T15:58:43.260 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 Processor -- start 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.320+0000 7f8e76528700 1 -- start start 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e76528700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e76528700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7019d6c0 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40076/0 (socket says 172.21.15.33:40076) 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6ffff700 1 -- 172.21.15.33:0/1506337454 learned_addr learned my addr 172.21.15.33:0/1506337454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6ffff700 1 -- 172.21.15.33:0/1506337454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60009550 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6ffff700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8e60003e60 tx=0x7f8e60011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e600038b0 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e60003a10 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e60019530 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.321+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7019d8c0 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.322+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7019dd60 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f8e6001a980 con 0x7f8e70105a90 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e6d7fa700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 0x7f8e5c03af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8e6004cb90 con 0x7f8e70105a90 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e70196e60 con 0x7f8e70105a90 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e6f7fe700 1 -- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 msgr2=0x7f8e5c03af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.323+0000 7f8e6f7fe700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 0x7f8e5c03af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.328+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8e60019690 con 0x7f8e70105a90 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.524+0000 7f8e6f7fe700 1 -- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 msgr2=0x7f8e5c03af60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.524+0000 7f8e6f7fe700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 0x7f8e5c03af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.592+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8e70107490 con 0x7f8e70105a90 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.593+0000 7f8e6d7fa700 1 -- 172.21.15.33:0/1506337454 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f8e60023b40 con 0x7f8e70105a90 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 msgr2=0x7f8e5c03af60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 0x7f8e5c03af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 msgr2=0x7f8e7019d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8e60003e60 tx=0x7f8e60011a30 comp rx=0 tx=0).stop 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 shutdown_connections 2024-01-25T15:58:43.262 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f8e5c038aa0 0x7f8e5c03af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 --2- 172.21.15.33:0/1506337454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e70105a90 0x7f8e7019d180 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 >> 172.21.15.33:0/1506337454 conn(0x7f8e70101710 msgr2=0x7f8e70193fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 shutdown_connections 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:42.595+0000 7f8e76528700 1 -- 172.21.15.33:0/1506337454 wait complete. 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-01-25T15:58:43.263 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-01-25T15:58:43.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1506337454' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-01-25T15:58:43.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:43.053+0000 7f95d561b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T15:58:43.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:43.237+0000 7f95d561b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:43.612 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:43.331+0000 7f95d561b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T15:58:43.612 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:43.428+0000 7f95d561b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T15:58:43.612 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:43.521+0000 7f95d561b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T15:58:44.938 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:44.603+0000 7f95d561b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T15:58:44.938 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:44.736+0000 7f95d561b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T15:58:44.939 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:44.935+0000 7f95d561b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T15:58:45.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:45.168+0000 7f95d561b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T15:58:45.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:45.352+0000 7f95d561b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:45.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:45.541+0000 7f95d561b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T15:58:46.308 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:45.816+0000 7f95d561b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T15:58:46.308 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:45.905+0000 7f95d561b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T15:58:46.564 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:46.305+0000 7f95d561b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T15:58:47.564 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:47.120+0000 7f95d561b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T15:58:48.008 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:47.700+0000 7f95d561b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T15:58:48.008 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:47.801+0000 7f95d561b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T15:58:48.008 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:58:48.005+0000 7f95d561b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T15:58:48.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: Active manager daemon x restarted 2024-01-25T15:58:48.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: Activating manager daemon x 2024-01-25T15:58:48.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: osdmap e3: 0 total, 0 up, 0 in 2024-01-25T15:58:48.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: mgrmap e10: x(active, starting, since 0.00340322s) 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: Manager daemon x is now available 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:48.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:58:49.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T15:58:49.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T15:58:49.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:49 smithi033 ceph-mon[31417]: mgrmap e11: x(active, since 1.00557s) 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.710+0000 7f34c46e0700 1 Processor -- start 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.710+0000 7f34c46e0700 1 -- start start 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c46e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c46e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc07b900 con 0x7f34bc07ce80 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c247c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:49.880 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c247c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40082/0 (socket says 172.21.15.33:40082) 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c247c700 1 -- 172.21.15.33:0/1452729055 learned_addr learned my addr 172.21.15.33:0/1452729055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c247c700 1 -- 172.21.15.33:0/1452729055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc07ba40 con 0x7f34bc07ce80 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.711+0000 7f34c247c700 1 --2- 172.21.15.33:0/1452729055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0098a0 tx=0x7f34ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f5c6a775b376fe server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c147a700 1 -- 172.21.15.33:0/1452729055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ac004160 con 0x7f34bc07ce80 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c147a700 1 -- 172.21.15.33:0/1452729055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac0042c0 con 0x7f34bc07ce80 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c147a700 1 -- 172.21.15.33:0/1452729055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ac0119b0 con 0x7f34bc07ce80 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 -- 172.21.15.33:0/1452729055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 msgr2=0x7f34bc07b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1452729055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0098a0 tx=0x7f34ac009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 -- 172.21.15.33:0/1452729055 shutdown_connections 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1452729055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc07b330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 -- 172.21.15.33:0/1452729055 >> 172.21.15.33:0/1452729055 conn(0x7f34bc1028a0 msgr2=0x7f34bc104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 -- 172.21.15.33:0/1452729055 shutdown_connections 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.712+0000 7f34c46e0700 1 -- 172.21.15.33:0/1452729055 wait complete. 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c46e0700 1 Processor -- start 2024-01-25T15:58:49.881 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c46e0700 1 -- start start 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c46e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c46e0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc19d680 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c247c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c247c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40086/0 (socket says 172.21.15.33:40086) 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.713+0000 7f34c247c700 1 -- 172.21.15.33:0/1905536409 learned_addr learned my addr 172.21.15.33:0/1905536409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34c247c700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac009550 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34c247c700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34ac011980 tx=0x7f34ac011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ac003af0 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34ac003c50 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc19d880 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ac019760 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.714+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc19dd20 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f34ac01a430 con 0x7f34bc07ce80 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34b37fe700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:49.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34a803b660 con 0x7f34a8038a90 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f34ac04cd20 con 0x7f34bc07ce80 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34c1c7b700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.715+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.916+0000 7f34c1c7b700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:43.916+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:44.317+0000 7f34c1c7b700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:44.317+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:45.118+0000 7f34c1c7b700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:45.118+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:46.719+0000 7f34c1c7b700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/2443786608 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:46.719+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:48.020+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) v1 ==== 45421+0+0 (secure 0 0 0) 0x7f34ac02bd40 con 0x7f34bc07ce80 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:48.020+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 msgr2=0x7f34a803af50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T15:58:49.883 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:48.020+0000 7f34b37fe700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.021+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f34ac04e750 con 0x7f34bc07ce80 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.021+0000 7f34b37fe700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 0x7f34a803e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.022+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34a803b660 con 0x7f34a803c500 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.022+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 0x7f34a803e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.023+0000 7f34c1c7b700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 0x7f34a803e8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f34b8003a10 tx=0x7f34b80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.023+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f34a803b660 con 0x7f34a803c500 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f34bc10d660 con 0x7f34a803c500 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34b37fe700 1 -- 172.21.15.33:0/1905536409 <== mgr.14152 v2:172.21.15.33:6800/1063041969 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f34bc10d660 con 0x7f34a803c500 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 msgr2=0x7f34a803e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 0x7f34a803e8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f34b8003a10 tx=0x7f34b80092b0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 msgr2=0x7f34bc19d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34ac011980 tx=0x7f34ac011a60 comp rx=0 tx=0).stop 2024-01-25T15:58:49.884 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 shutdown_connections 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f34a803c500 0x7f34a803e8f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34bc07ce80 0x7f34bc19d140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 --2- 172.21.15.33:0/1905536409 >> [v2:172.21.15.33:6800/2443786608,v1:172.21.15.33:6801/2443786608] conn(0x7f34a8038a90 0x7f34a803af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 >> 172.21.15.33:0/1905536409 conn(0x7f34bc1028a0 msgr2=0x7f34bc103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.030+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 shutdown_connections 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:49.031+0000 7f34c46e0700 1 -- 172.21.15.33:0/1905536409 wait complete. 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-01-25T15:58:49.885 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:48] ENGINE Bus STARTING 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:48] ENGINE Serving on https://172.21.15.33:7150 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:49] ENGINE Serving on http://172.21.15.33:8765 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: [25/Jan/2024:15:58:49] ENGINE Bus STARTED 2024-01-25T15:58:50.250 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:58:51.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-01-25T15:58:51.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1286d8e700 1 Processor -- start 2024-01-25T15:58:51.201 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1286d8e700 1 -- start start 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1286d8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1286d8e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128007aec0 con 0x7f1280107cb0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1284b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1284b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47650/0 (socket says 172.21.15.33:47650) 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.320+0000 7f1284b2a700 1 -- 172.21.15.33:0/3643572998 learned_addr learned my addr 172.21.15.33:0/3643572998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.321+0000 7f1284b2a700 1 -- 172.21.15.33:0/3643572998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128010a5e0 con 0x7f1280107cb0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.321+0000 7f1284b2a700 1 --2- 172.21.15.33:0/3643572998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36194c0b953a989d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.321+0000 7f127f7fe700 1 -- 172.21.15.33:0/3643572998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1274004160 con 0x7f1280107cb0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.321+0000 7f127f7fe700 1 -- 172.21.15.33:0/3643572998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f12740042c0 con 0x7f1280107cb0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.321+0000 7f127f7fe700 1 -- 172.21.15.33:0/3643572998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12740119b0 con 0x7f1280107cb0 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- 172.21.15.33:0/3643572998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 msgr2=0x7f128010a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 --2- 172.21.15.33:0/3643572998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f12740098a0 tx=0x7f1274009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- 172.21.15.33:0/3643572998 shutdown_connections 2024-01-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 --2- 172.21.15.33:0/3643572998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128010a0a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- 172.21.15.33:0/3643572998 >> 172.21.15.33:0/3643572998 conn(0x7f1280101720 msgr2=0x7f1280103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- 172.21.15.33:0/3643572998 shutdown_connections 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- 172.21.15.33:0/3643572998 wait complete. 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 Processor -- start 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- start start 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.322+0000 7f1286d8e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128019d6f0 con 0x7f1280107cb0 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1284b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1284b2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47666/0 (socket says 172.21.15.33:47666) 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1284b2a700 1 -- 172.21.15.33:0/155675102 learned_addr learned my addr 172.21.15.33:0/155675102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1284b2a700 1 -- 172.21.15.33:0/155675102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1274009550 con 0x7f1280107cb0 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1284b2a700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1274003e60 tx=0x7f1274011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12740038b0 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1274003a10 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128019d8f0 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1274019530 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.323+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128019dd90 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.324+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 45548+0+0 (secure 0 0 0) 0x7f12740197f0 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.324+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128010b420 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.324+0000 7f127dffb700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 0x7f127003adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.324+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f127404dab0 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.325+0000 7f127ffff700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 0x7f127003adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.325+0000 7f127ffff700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 0x7f127003adf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f126c006fd0 tx=0x7f126c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.329+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f127402b340 con 0x7f1280107cb0 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.541+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f12801a0670 con 0x7f1270038930 2024-01-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.682+0000 7f127dffb700 1 -- 172.21.15.33:0/155675102 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f12801a0670 con 0x7f1270038930 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 msgr2=0x7f127003adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 0x7f127003adf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f126c006fd0 tx=0x7f126c006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 msgr2=0x7f128019d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1274003e60 tx=0x7f1274011a30 comp rx=0 tx=0).stop 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 shutdown_connections 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1270038930 0x7f127003adf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 --2- 172.21.15.33:0/155675102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1280107cb0 0x7f128019d1b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 >> 172.21.15.33:0/155675102 conn(0x7f1280101720 msgr2=0x7f12801021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 shutdown_connections 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:50.685+0000 7f1286d8e700 1 -- 172.21.15.33:0/155675102 wait complete. 2024-01-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-01-25T15:58:51.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:51 smithi033 ceph-mon[31417]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:51.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:58:51.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:58:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:58:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:51 smithi033 ceph-mon[31417]: mgrmap e12: x(active, since 2s) 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$yzyYf0U.QCEClUr2vGouoOZ7KZ1tVMbfKsE9kni8Cw9BhRgHST5jG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1706198332, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.630+0000 7f7228ac6700 1 Processor -- start 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.630+0000 7f7228ac6700 1 -- start start 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f7228ac6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f7228ac6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224105f00 con 0x7f7224105510 2024-01-25T15:58:52.812 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f722259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f722259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47674/0 (socket says 172.21.15.33:47674) 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f722259c700 1 -- 172.21.15.33:0/2258353856 learned_addr learned my addr 172.21.15.33:0/2258353856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f722259c700 1 -- 172.21.15.33:0/2258353856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72241084a0 con 0x7f7224105510 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.631+0000 7f722259c700 1 --2- 172.21.15.33:0/2258353856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72140098a0 tx=0x7f7214009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b121a649ebd4b246 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.632+0000 7f722159a700 1 -- 172.21.15.33:0/2258353856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7214004160 con 0x7f7224105510 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.632+0000 7f722159a700 1 -- 172.21.15.33:0/2258353856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72140042c0 con 0x7f7224105510 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.632+0000 7f722159a700 1 -- 172.21.15.33:0/2258353856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72140119b0 con 0x7f7224105510 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.632+0000 7f7228ac6700 1 -- 172.21.15.33:0/2258353856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 msgr2=0x7f7224105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.632+0000 7f7228ac6700 1 --2- 172.21.15.33:0/2258353856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f72140098a0 tx=0x7f7214009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 -- 172.21.15.33:0/2258353856 shutdown_connections 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 --2- 172.21.15.33:0/2258353856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224105930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 -- 172.21.15.33:0/2258353856 >> 172.21.15.33:0/2258353856 conn(0x7f7224100ee0 msgr2=0x7f7224103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:52.813 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 -- 172.21.15.33:0/2258353856 shutdown_connections 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 -- 172.21.15.33:0/2258353856 wait complete. 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.633+0000 7f7228ac6700 1 Processor -- start 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f7228ac6700 1 -- start start 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f7228ac6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f7228ac6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224108260 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f722259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f722259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47686/0 (socket says 172.21.15.33:47686) 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f722259c700 1 -- 172.21.15.33:0/3086411722 learned_addr learned my addr 172.21.15.33:0/3086411722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f722259c700 1 -- 172.21.15.33:0/3086411722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009550 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.634+0000 7f722259c700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7214011670 tx=0x7f7214011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.635+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7214003af0 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.635+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7214003c50 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.635+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7224106450 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.635+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7214024820 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.635+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72241068f0 con 0x7f7224105510 2024-01-25T15:58:52.814 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.636+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7f7214024a50 con 0x7f7224105510 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.636+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722410b6d0 con 0x7f7224105510 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.636+0000 7f72137fe700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 0x7f720c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.636+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f721404cc50 con 0x7f7224105510 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.636+0000 7f7221d9b700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 0x7f720c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.637+0000 7f7221d9b700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 0x7f720c03afb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7218006fd0 tx=0x7f7218006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.641+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7214010790 con 0x7f7224105510 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:51.860+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7224062c30 con 0x7f720c038af0 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.107+0000 7f72137fe700 1 -- 172.21.15.33:0/3086411722 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7224062c30 con 0x7f720c038af0 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 msgr2=0x7f720c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 0x7f720c03afb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7218006fd0 tx=0x7f7218006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 msgr2=0x7f7224107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7214011670 tx=0x7f7214011a30 comp rx=0 tx=0).stop 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 shutdown_connections 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f720c038af0 0x7f720c03afb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:52.815 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 --2- 172.21.15.33:0/3086411722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7224105510 0x7f7224107d20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:52.816 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 >> 172.21.15.33:0/3086411722 conn(0x7f7224100ee0 msgr2=0x7f7224101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:52.816 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 shutdown_connections 2024-01-25T15:58:52.816 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:52.109+0000 7f7228ac6700 1 -- 172.21.15.33:0/3086411722 wait complete. 2024-01-25T15:58:52.816 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-01-25T15:58:53.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:53 smithi033 ceph-mon[31417]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:58:53.424 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:58:54.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:54 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2299605017' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.211+0000 7fe5cceea700 1 Processor -- start 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.211+0000 7fe5cceea700 1 -- start start 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5cceea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5cceea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8106b00 con 0x7fe5c8106110 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47690/0 (socket says 172.21.15.33:47690) 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5c659c700 1 -- 172.21.15.33:0/1124762936 learned_addr learned my addr 172.21.15.33:0/1124762936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5c659c700 1 -- 172.21.15.33:0/1124762936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8107310 con 0x7fe5c8106110 2024-01-25T15:58:54.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.212+0000 7fe5c659c700 1 --2- 172.21.15.33:0/1124762936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800bb40 tx=0x7fe5b800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8827abd9286a0eee server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5c559a700 1 -- 172.21.15.33:0/1124762936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5b800dbe0 con 0x7fe5c8106110 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5c559a700 1 -- 172.21.15.33:0/1124762936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b80044f0 con 0x7fe5c8106110 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5c559a700 1 -- 172.21.15.33:0/1124762936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8003e60 con 0x7fe5c8106110 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 -- 172.21.15.33:0/1124762936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 msgr2=0x7fe5c8106530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 --2- 172.21.15.33:0/1124762936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800bb40 tx=0x7fe5b800be50 comp rx=0 tx=0).stop 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 -- 172.21.15.33:0/1124762936 shutdown_connections 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 --2- 172.21.15.33:0/1124762936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c8106530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 -- 172.21.15.33:0/1124762936 >> 172.21.15.33:0/1124762936 conn(0x7fe5c81016d0 msgr2=0x7fe5c8103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 -- 172.21.15.33:0/1124762936 shutdown_connections 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.213+0000 7fe5cceea700 1 -- 172.21.15.33:0/1124762936 wait complete. 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5cceea700 1 Processor -- start 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5cceea700 1 -- start start 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5cceea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5cceea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c819d680 con 0x7fe5c8106110 2024-01-25T15:58:54.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47702/0 (socket says 172.21.15.33:47702) 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.214+0000 7fe5c659c700 1 -- 172.21.15.33:0/2299605017 learned_addr learned my addr 172.21.15.33:0/2299605017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5c659c700 1 -- 172.21.15.33:0/2299605017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b800b7f0 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5c659c700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80104d0 tx=0x7fe5b8004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8010760 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c819d880 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5b8010d80 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5b8019ce0 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.215+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c819dcc0 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.216+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 45654+0+0 (secure 0 0 0) 0x7fe5b802a450 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.216+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8079fe0 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.216+0000 7fe5b77fe700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 0x7fe5b003af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.216+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe5b804ceb0 con 0x7fe5c8106110 2024-01-25T15:58:54.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.216+0000 7fe5c5d9b700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 0x7fe5b003af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.217+0000 7fe5c5d9b700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 0x7fe5b003af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc006fd0 tx=0x7fe5bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.221+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe5b8028710 con 0x7fe5c8106110 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.421+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fe5c8192410 con 0x7fe5c8106110 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.421+0000 7fe5b77fe700 1 -- 172.21.15.33:0/2299605017 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fe5b804e0b0 con 0x7fe5c8106110 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 msgr2=0x7fe5b003af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 0x7fe5b003af00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc006fd0 tx=0x7fe5bc006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 msgr2=0x7fe5c819d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80104d0 tx=0x7fe5b8004020 comp rx=0 tx=0).stop 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 shutdown_connections 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe5b0038a40 0x7fe5b003af00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 --2- 172.21.15.33:0/2299605017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5c8106110 0x7fe5c819d140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.423+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 >> 172.21.15.33:0/2299605017 conn(0x7fe5c81016d0 msgr2=0x7fe5c818f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.424+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 shutdown_connections 2024-01-25T15:58:54.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:53.424+0000 7fe5cceea700 1 -- 172.21.15.33:0/2299605017 wait complete. 2024-01-25T15:58:54.314 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-01-25T15:58:54.314 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-01-25T15:58:54.322 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-01-25T15:58:54.322 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-01-25T15:58:54.322 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-01-25T15:58:54.323 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033.front.sepia.ceph.com:8443/ 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout: Password: z4w04fn0jr 2024-01-25T15:58:54.324 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:54.325 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config directory 2024-01-25T15:58:54.325 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-01-25T15:58:55.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:55 smithi033 ceph-mon[31417]: mgrmap e13: x(active, since 6s) 2024-01-25T15:58:55.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.766+0000 7fab3117a700 1 Processor -- start 2024-01-25T15:58:55.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.766+0000 7fab3117a700 1 -- start start 2024-01-25T15:58:55.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.766+0000 7fab3117a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:55.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.766+0000 7fab3117a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c07ae60 con 0x7fab2c107ce0 2024-01-25T15:58:55.587 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47706/0 (socket says 172.21.15.33:47706) 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab2ad9d700 1 -- 172.21.15.33:0/591511917 learned_addr learned my addr 172.21.15.33:0/591511917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab2ad9d700 1 -- 172.21.15.33:0/591511917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c10a610 con 0x7fab2c107ce0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/591511917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0098a0 tx=0x7fab1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c52a1f438c4b5f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab29d9b700 1 -- 172.21.15.33:0/591511917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c004160 con 0x7fab2c107ce0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab29d9b700 1 -- 172.21.15.33:0/591511917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c0042c0 con 0x7fab2c107ce0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.767+0000 7fab29d9b700 1 -- 172.21.15.33:0/591511917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c0119b0 con 0x7fab2c107ce0 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 -- 172.21.15.33:0/591511917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 msgr2=0x7fab2c10a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 --2- 172.21.15.33:0/591511917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0098a0 tx=0x7fab1c009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 -- 172.21.15.33:0/591511917 shutdown_connections 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 --2- 172.21.15.33:0/591511917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c10a0d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 -- 172.21.15.33:0/591511917 >> 172.21.15.33:0/591511917 conn(0x7fab2c1016d0 msgr2=0x7fab2c103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 -- 172.21.15.33:0/591511917 shutdown_connections 2024-01-25T15:58:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.768+0000 7fab3117a700 1 -- 172.21.15.33:0/591511917 wait complete. 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab3117a700 1 Processor -- start 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab3117a700 1 -- start start 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab3117a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab3117a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c107a50 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab2ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47710/0 (socket says 172.21.15.33:47710) 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.769+0000 7fab2ad9d700 1 -- 172.21.15.33:0/2299812257 learned_addr learned my addr 172.21.15.33:0/2299812257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab2ad9d700 1 -- 172.21.15.33:0/2299812257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c009550 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab2ad9d700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006af0 tx=0x7fab1c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c003a40 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2c105c90 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2c106130 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab1c003ba0 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.770+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab1c019670 con 0x7fab2c107ce0 2024-01-25T15:58:55.589 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.771+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fab1c019930 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.771+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18005360 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.771+0000 7fab13fff700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 0x7fab1403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.771+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fab1c04d140 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.771+0000 7fab2a59c700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 0x7fab1403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.772+0000 7fab2a59c700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 0x7fab1403afa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.776+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fab1c02b750 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.978+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fab18005610 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.979+0000 7fab13fff700 1 -- 172.21.15.33:0/2299812257 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fab1c023390 con 0x7fab2c107ce0 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 msgr2=0x7fab1403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 0x7fab1403afa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fab20006fd0 tx=0x7fab20006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 msgr2=0x7fab2c107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab1c006af0 tx=0x7fab1c011a10 comp rx=0 tx=0).stop 2024-01-25T15:58:55.590 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 shutdown_connections 2024-01-25T15:58:55.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fab14038ae0 0x7fab1403afa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:55.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 --2- 172.21.15.33:0/2299812257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c107ce0 0x7fab2c107510 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:55.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.981+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 >> 172.21.15.33:0/2299812257 conn(0x7fab2c1016d0 msgr2=0x7fab2c1021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:55.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.982+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 shutdown_connections 2024-01-25T15:58:55.591 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:54.982+0000 7fab3117a700 1 -- 172.21.15.33:0/2299812257 wait complete. 2024-01-25T15:58:56.901 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.997+0000 7f60c7515700 1 Processor -- start 2024-01-25T15:58:56.901 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.997+0000 7f60c7515700 1 -- start start 2024-01-25T15:58:56.901 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.997+0000 7f60c7515700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:56.901 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.997+0000 7f60c7515700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c0107bc0 con 0x7f60c01071d0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60c52b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60c52b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47722/0 (socket says 172.21.15.33:47722) 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60c52b1700 1 -- 172.21.15.33:0/3174577326 learned_addr learned my addr 172.21.15.33:0/3174577326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60c52b1700 1 -- 172.21.15.33:0/3174577326 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c01083d0 con 0x7f60c01071d0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60c52b1700 1 --2- 172.21.15.33:0/3174577326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60b80098a0 tx=0x7f60b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b001893daa8cb8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60bffff700 1 -- 172.21.15.33:0/3174577326 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b8004160 con 0x7f60c01071d0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60bffff700 1 -- 172.21.15.33:0/3174577326 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b80042c0 con 0x7f60c01071d0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.998+0000 7f60bffff700 1 -- 172.21.15.33:0/3174577326 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b80119b0 con 0x7f60c01071d0 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 -- 172.21.15.33:0/3174577326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 msgr2=0x7f60c01075f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 --2- 172.21.15.33:0/3174577326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f60b80098a0 tx=0x7f60b8009bb0 comp rx=0 tx=0).stop 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 -- 172.21.15.33:0/3174577326 shutdown_connections 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 --2- 172.21.15.33:0/3174577326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c01075f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 -- 172.21.15.33:0/3174577326 >> 172.21.15.33:0/3174577326 conn(0x7f60c01027f0 msgr2=0x7f60c0104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 -- 172.21.15.33:0/3174577326 shutdown_connections 2024-01-25T15:58:56.902 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:55.999+0000 7f60c7515700 1 -- 172.21.15.33:0/3174577326 wait complete. 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c7515700 1 Processor -- start 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c7515700 1 -- start start 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c7515700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c7515700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c019d660 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c52b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c52b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47728/0 (socket says 172.21.15.33:47728) 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c52b1700 1 -- 172.21.15.33:0/2847626919 learned_addr learned my addr 172.21.15.33:0/2847626919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c52b1700 1 -- 172.21.15.33:0/2847626919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b8009550 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c52b1700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006af0 tx=0x7f60b8011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b8003a40 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60b8003ba0 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.000+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c019d860 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.001+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c019dd00 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.001+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60b8019690 con 0x7f60c01071d0 2024-01-25T15:58:56.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.001+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f60b8019900 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.001+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c010b600 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.002+0000 7f60be7fc700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 0x7f60a803b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.002+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60b804d020 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.002+0000 7f60c4ab0700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 0x7f60a803b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.002+0000 7f60c4ab0700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 0x7f60a803b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60b4006fd0 tx=0x7f60b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.007+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f60b8080600 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.322+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f60c01a05e0 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.324+0000 7f60be7fc700 1 -- 172.21.15.33:0/2847626919 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f60b8023b80 con 0x7f60c01071d0 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 msgr2=0x7f60a803b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 0x7f60a803b000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f60b4006fd0 tx=0x7f60b4006e40 comp rx=0 tx=0).stop 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 msgr2=0x7f60c019d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f60b8006af0 tx=0x7f60b8011a10 comp rx=0 tx=0).stop 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 shutdown_connections 2024-01-25T15:58:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f60a8038b40 0x7f60a803b000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 --2- 172.21.15.33:0/2847626919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60c01071d0 0x7f60c019d120 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 >> 172.21.15.33:0/2847626919 conn(0x7f60c01027f0 msgr2=0x7f60c0103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 shutdown_connections 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-01-25T15:58:56.326+0000 7f60c7515700 1 -- 172.21.15.33:0/2847626919 wait complete. 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.905 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:58:56.906 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-01-25T15:58:56.944 INFO:tasks.cephadm:Fetching config... 2024-01-25T15:58:56.944 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:58:56.944 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-01-25T15:58:56.960 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-01-25T15:58:56.960 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:58:56.960 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-01-25T15:58:57.015 INFO:tasks.cephadm:Fetching mon keyring... 2024-01-25T15:58:57.015 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:58:57.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/keyring of=/dev/stdout 2024-01-25T15:58:57.082 INFO:tasks.cephadm:Fetching pub ssh key... 2024-01-25T15:58:57.083 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:58:57.083 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-01-25T15:58:57.137 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-01-25T15:58:57.137 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-25T15:58:57.214 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:57.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-25T15:58:57.274 INFO:teuthology.orchestra.run.smithi076.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:57.290 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-01-25T15:58:57.336 INFO:teuthology.orchestra.run.smithi152.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDQ+QzwsQG/GDDKZhtJ4k1KTFy4ZPPb6Jcs2uRU+0g8XR0A5y7tJdb+2/EgXJ2J7fwLCoOOaZW9mzdsuww7O0/3eYZ2UiHi503UAC8bcf8tTcnlZliQ3wynpQset7mf+fGvAxPsGvLeNDu7/BDlg7y92wKEF3AtbVmEPF28tb0BMBddes9tB6gYm1SjNC77r58/lZxrqFlluW2mHpFmpx5V9g3dM649ReGTPB9u6GrAeVhpWxgKYfhltcAC/EnVwfRqVj+Ej5C6E5IU6QwVF4bA/S+vp3aCUQVaKQSpe3Yb+KOhHRkixvqgvBAPWcAIAK1MaXPoau7I41Pqt0XdGij8iKwIMgYrrCt8Tbb9CO0PtoULzy5SDd9xLBcyz809yo7JTa21HYhJYYotji1yMkg7lCV36YdnCHTAVELEvtpLeg1DlFNJ1JUta9jJb0ZyiXvgF1gHm1GiLcxaoICpX2cU5hst8TC9CuFinuYDDaxORxr8tA5YiiAn3EIpWPW8xKE= ceph-55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:58:57.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-01-25T15:58:57.439 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:58:57 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2847626919' entity='client.admin' 2024-01-25T15:58:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 -- 172.21.15.33:0/3106970046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 msgr2=0x7f67c0100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 --2- 172.21.15.33:0/3106970046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c0100530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009910 tx=0x7f67b0009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 -- 172.21.15.33:0/3106970046 shutdown_connections 2024-01-25T15:59:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 --2- 172.21.15.33:0/3106970046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c0100530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 -- 172.21.15.33:0/3106970046 >> 172.21.15.33:0/3106970046 conn(0x7f67c00fb6b0 msgr2=0x7f67c00fdaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 -- 172.21.15.33:0/3106970046 shutdown_connections 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.041+0000 7f67c626a700 1 -- 172.21.15.33:0/3106970046 wait complete. 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67c626a700 1 Processor -- start 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67c626a700 1 -- start start 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67c626a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67c626a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c018d650 con 0x7f67c0100110 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38048/0 (socket says 172.21.15.33:38048) 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.042+0000 7f67bffff700 1 -- 172.21.15.33:0/2463027522 learned_addr learned my addr 172.21.15.33:0/2463027522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67bffff700 1 -- 172.21.15.33:0/2463027522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b00095f0 con 0x7f67c0100110 2024-01-25T15:59:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67bffff700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005950 tx=0x7f67b00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b001d070 con 0x7f67c0100110 2024-01-25T15:59:00.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b000bc10 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67b0003a10 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c018b840 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.043+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c018bce0 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.044+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f67b0003cd0 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.044+0000 7f67bd7fa700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 0x7f67a803afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.044+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c0062e10 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.044+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f67b004c8b0 con 0x7f67c0100110 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.044+0000 7f67bf7fe700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 0x7f67a803afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:00.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.045+0000 7f67bf7fe700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 0x7f67a803afa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006fd0 tx=0x7f67b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:00.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.049+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f67b000f460 con 0x7f67c0100110 2024-01-25T15:59:00.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.252+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f67c0106020 con 0x7f67c0100110 2024-01-25T15:59:00.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.256+0000 7f67bd7fa700 1 -- 172.21.15.33:0/2463027522 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f67b0029360 con 0x7f67c0100110 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 msgr2=0x7f67a803afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 0x7f67a803afa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f67b4006fd0 tx=0x7f67b4006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 msgr2=0x7f67c018d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f67b0005950 tx=0x7f67b00057c0 comp rx=0 tx=0).stop 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 shutdown_connections 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67a8038ae0 0x7f67a803afa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 --2- 172.21.15.33:0/2463027522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c0100110 0x7f67c018d110 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 >> 172.21.15.33:0/2463027522 conn(0x7f67c00fb6b0 msgr2=0x7f67c0069460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 shutdown_connections 2024-01-25T15:59:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:00.259+0000 7f67c626a700 1 -- 172.21.15.33:0/2463027522 wait complete. 2024-01-25T15:59:01.413 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-01-25T15:59:01.414 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-01-25T15:59:01.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2463027522' entity='client.admin' 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:01.546 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/4226222913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 msgr2=0x7fd19c100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/4226222913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c100550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd190009910 tx=0x7fd190009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/4226222913 shutdown_connections 2024-01-25T15:59:03.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/4226222913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c100550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/4226222913 >> 172.21.15.33:0/4226222913 conn(0x7fd19c0fb6f0 msgr2=0x7fd19c0fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/4226222913 shutdown_connections 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.011+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/4226222913 wait complete. 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a2cd2700 1 Processor -- start 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a2cd2700 1 -- start start 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a2cd2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a2cd2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c195ad0 con 0x7fd19c100130 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a0a6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a0a6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38064/0 (socket says 172.21.15.33:38064) 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a0a6e700 1 -- 172.21.15.33:0/3866660930 learned_addr learned my addr 172.21.15.33:0/3866660930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a0a6e700 1 -- 172.21.15.33:0/3866660930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1900095f0 con 0x7fd19c100130 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd1a0a6e700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd190004f70 tx=0x7fd190004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:03.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.012+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd19001d070 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.013+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd19000be20 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.013+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c195cd0 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.013+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd190022df0 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.013+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c196170 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fd19000b960 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c18ee50 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd199ffb700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 0x7fd18c03b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd19004ccf0 con 0x7fd19c100130 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd19bfff700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 0x7fd18c03b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.014+0000 7fd19bfff700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 0x7fd18c03b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd188006fd0 tx=0x7fd188006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.019+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd19000e3e0 con 0x7fd19c100130 2024-01-25T15:59:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.221+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd19c02d4a0 con 0x7fd18c038b40 2024-01-25T15:59:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.224+0000 7fd199ffb700 1 -- 172.21.15.33:0/3866660930 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd19c02d4a0 con 0x7fd18c038b40 2024-01-25T15:59:03.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 msgr2=0x7fd18c03b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 0x7fd18c03b000 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd188006fd0 tx=0x7fd188006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 msgr2=0x7fd19c195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd190004f70 tx=0x7fd190004fa0 comp rx=0 tx=0).stop 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 shutdown_connections 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd18c038b40 0x7fd18c03b000 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 --2- 172.21.15.33:0/3866660930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd19c100130 0x7fd19c195590 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 >> 172.21.15.33:0/3866660930 conn(0x7fd19c0fb6f0 msgr2=0x7fd19c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 shutdown_connections 2024-01-25T15:59:03.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:03.227+0000 7fd1a2cd2700 1 -- 172.21.15.33:0/3866660930 wait complete. 2024-01-25T15:59:03.992 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi076 2024-01-25T15:59:03.992 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:59:03.992 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.conf 2024-01-25T15:59:04.011 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:59:04.011 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:04.069 INFO:tasks.cephadm:Adding host smithi076 to orchestrator... 2024-01-25T15:59:04.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch host add smithi076 2024-01-25T15:59:04.307 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: Updating smithi033:/etc/ceph/ceph.conf 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:04.520 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:04.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T15:59:04.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:04.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:04.521 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:05.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- 172.21.15.33:0/1402779728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 msgr2=0x7f5270102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 --2- 172.21.15.33:0/1402779728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f5270102790 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5260009910 tx=0x7f5260009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- 172.21.15.33:0/1402779728 shutdown_connections 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 --2- 172.21.15.33:0/1402779728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f5270102790 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- 172.21.15.33:0/1402779728 >> 172.21.15.33:0/1402779728 conn(0x7f52700fbc20 msgr2=0x7f52700fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- 172.21.15.33:0/1402779728 shutdown_connections 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- 172.21.15.33:0/1402779728 wait complete. 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 Processor -- start 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.792+0000 7f527662b700 1 -- start start 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f527662b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f527662b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527019cc40 con 0x7f52701023b0 2024-01-25T15:59:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38068/0 (socket says 172.21.15.33:38068) 2024-01-25T15:59:05.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526ffff700 1 -- 172.21.15.33:0/3936102227 learned_addr learned my addr 172.21.15.33:0/3936102227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526ffff700 1 -- 172.21.15.33:0/3936102227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600095f0 con 0x7f52701023b0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526ffff700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5260004f40 tx=0x7f5260005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f526001d070 con 0x7f52701023b0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.793+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5260004370 con 0x7f52701023b0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527019ce40 con 0x7f52701023b0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5260003970 con 0x7f52701023b0 2024-01-25T15:59:05.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527019d260 con 0x7f52701023b0 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f5260003c30 con 0x7f52701023b0 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f526d7fa700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 0x7f525803afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.794+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5270062e10 con 0x7f52701023b0 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.795+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f526004c910 con 0x7f52701023b0 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.795+0000 7f526f7fe700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 0x7f525803afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:05.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.795+0000 7f526f7fe700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 0x7f525803afa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5264006fd0 tx=0x7f5264006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:05.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:05.800+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f526002aba0 con 0x7f52701023b0 2024-01-25T15:59:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:06.002+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}) v1 -- 0x7f527010b650 con 0x7f5258038ae0 2024-01-25T15:59:06.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:06 smithi033 ceph-mon[31417]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi076", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:07 smithi033 ceph-mon[31417]: Deploying cephadm binary to smithi076 2024-01-25T15:59:09.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:09 smithi033 ceph-mon[31417]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:10.231 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi076' with addr '172.21.15.76' 2024-01-25T15:59:10.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.228+0000 7f526d7fa700 1 -- 172.21.15.33:0/3936102227 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f527010b650 con 0x7f5258038ae0 2024-01-25T15:59:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 msgr2=0x7f525803afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 0x7f525803afa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5264006fd0 tx=0x7f5264006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 msgr2=0x7f527019c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:10.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5260004f40 tx=0x7f5260005020 comp rx=0 tx=0).stop 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 shutdown_connections 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5258038ae0 0x7f525803afa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 --2- 172.21.15.33:0/3936102227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52701023b0 0x7f527019c700 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 >> 172.21.15.33:0/3936102227 conn(0x7f52700fbc20 msgr2=0x7f5270103160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.230+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 shutdown_connections 2024-01-25T15:59:10.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:10.231+0000 7f527662b700 1 -- 172.21.15.33:0/3936102227 wait complete. 2024-01-25T15:59:10.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch host ls --format=json 2024-01-25T15:59:11.095 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:11.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:10 smithi033 ceph-mon[31417]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:11.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:11.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:11.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:12.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:11 smithi033 ceph-mon[31417]: Added host smithi076 2024-01-25T15:59:12.232 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:11 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:12.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- 172.21.15.33:0/1314414564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 msgr2=0x7efde4101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:12.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 --2- 172.21.15.33:0/1314414564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde4101800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efde0009910 tx=0x7efde0009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:12.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- 172.21.15.33:0/1314414564 shutdown_connections 2024-01-25T15:59:12.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 --2- 172.21.15.33:0/1314414564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde4101800 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- 172.21.15.33:0/1314414564 >> 172.21.15.33:0/1314414564 conn(0x7efde40fccf0 msgr2=0x7efde40ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- 172.21.15.33:0/1314414564 shutdown_connections 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- 172.21.15.33:0/1314414564 wait complete. 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 Processor -- start 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.735+0000 7efdeb764700 1 -- start start 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdeb764700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdeb764700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde419cc10 con 0x7efde4101420 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efde9500700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efde9500700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57968/0 (socket says 172.21.15.33:57968) 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efde9500700 1 -- 172.21.15.33:0/1803514468 learned_addr learned my addr 172.21.15.33:0/1803514468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efde9500700 1 -- 172.21.15.33:0/1803514468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00095f0 con 0x7efde4101420 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efde9500700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efde0004f40 tx=0x7efde0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efde001d070 con 0x7efde4101420 2024-01-25T15:59:12.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde419ce10 con 0x7efde4101420 2024-01-25T15:59:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efde000bc10 con 0x7efde4101420 2024-01-25T15:59:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efde00037d0 con 0x7efde4101420 2024-01-25T15:59:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.736+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde419d230 con 0x7efde4101420 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.737+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7efde0003a90 con 0x7efde4101420 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.737+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde4062e10 con 0x7efde4101420 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.737+0000 7efdda7fc700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 0x7efdd003afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.737+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efde004c880 con 0x7efde4101420 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.738+0000 7efde8cff700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 0x7efdd003afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.738+0000 7efde8cff700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 0x7efdd003afa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdd4006fd0 tx=0x7efdd4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.742+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7efde002aa00 con 0x7efde4101420 2024-01-25T15:59:12.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.948+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efde410b620 con 0x7efdd0038ae0 2024-01-25T15:59:12.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.949+0000 7efdda7fc700 1 -- 172.21.15.33:0/1803514468 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7efde410b620 con 0x7efdd0038ae0 2024-01-25T15:59:12.952 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:59:12.952 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}] 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 msgr2=0x7efdd003afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 0x7efdd003afa0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdd4006fd0 tx=0x7efdd4006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 msgr2=0x7efde419c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efde0004f40 tx=0x7efde0005020 comp rx=0 tx=0).stop 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 shutdown_connections 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdd0038ae0 0x7efdd003afa0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 --2- 172.21.15.33:0/1803514468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efde4101420 0x7efde419c6d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 >> 172.21.15.33:0/1803514468 conn(0x7efde40fccf0 msgr2=0x7efde4102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 shutdown_connections 2024-01-25T15:59:12.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:12.951+0000 7efdeb764700 1 -- 172.21.15.33:0/1803514468 wait complete. 2024-01-25T15:59:13.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:12 smithi033 ceph-mon[31417]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:13.296 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:13.574 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi152 2024-01-25T15:59:13.574 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:59:13.574 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.conf 2024-01-25T15:59:13.594 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:59:13.594 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:13.650 INFO:tasks.cephadm:Adding host smithi152 to orchestrator... 2024-01-25T15:59:13.651 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch host add smithi152 2024-01-25T15:59:13.887 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:14.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:13 smithi033 ceph-mon[31417]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T15:59:15.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:14 smithi033 ceph-mon[31417]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 -- 172.21.15.33:0/2770423594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 msgr2=0x7ff6881006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 --2- 172.21.15.33:0/2770423594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff6881006f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff678009910 tx=0x7ff678009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 -- 172.21.15.33:0/2770423594 shutdown_connections 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 --2- 172.21.15.33:0/2770423594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff6881006f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 -- 172.21.15.33:0/2770423594 >> 172.21.15.33:0/2770423594 conn(0x7ff688076d20 msgr2=0x7ff6880fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:15.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 -- 172.21.15.33:0/2770423594 shutdown_connections 2024-01-25T15:59:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.351+0000 7ff68def8700 1 -- 172.21.15.33:0/2770423594 wait complete. 2024-01-25T15:59:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff68def8700 1 Processor -- start 2024-01-25T15:59:15.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff68def8700 1 -- start start 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff68def8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff68def8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68819bb60 con 0x7ff688100310 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff6877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff6877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57980/0 (socket says 172.21.15.33:57980) 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff6877fe700 1 -- 172.21.15.33:0/3924440449 learned_addr learned my addr 172.21.15.33:0/3924440449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff6877fe700 1 -- 172.21.15.33:0/3924440449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780095f0 con 0x7ff688100310 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.352+0000 7ff6877fe700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff678006010 tx=0x7ff678004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff67801d070 con 0x7ff688100310 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68819bd60 con 0x7ff688100310 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff67800bc10 con 0x7ff688100310 2024-01-25T15:59:15.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780038a0 con 0x7ff688100310 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68819c180 con 0x7ff688100310 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.353+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7ff678003ad0 con 0x7ff688100310 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.354+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6881017f0 con 0x7ff688100310 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.354+0000 7ff6857fa700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 0x7ff67403afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.354+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff67804c750 con 0x7ff688100310 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.354+0000 7ff67effd700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 0x7ff67403afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:15.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.354+0000 7ff67effd700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 0x7ff67403afa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:15.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.359+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff67802ab60 con 0x7ff688100310 2024-01-25T15:59:15.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:15.567+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}) v1 -- 0x7ff68810a4e0 con 0x7ff674038ae0 2024-01-25T15:59:16.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:16.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi076", "name": "osd_memory_target"}]: dispatch 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: Updating smithi076:/etc/ceph/ceph.conf 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi152", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:16.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:16 smithi033 ceph-mon[31417]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:18.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:17 smithi033 ceph-mon[31417]: Deploying cephadm binary to smithi152 2024-01-25T15:59:18.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:17 smithi033 ceph-mon[31417]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T15:59:18.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:18.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:18.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:19.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:18 smithi033 ceph-mon[31417]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:19.840 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi152' with addr '172.21.15.152' 2024-01-25T15:59:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.836+0000 7ff6857fa700 1 -- 172.21.15.33:0/3924440449 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff68810a4e0 con 0x7ff674038ae0 2024-01-25T15:59:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.838+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 msgr2=0x7ff67403afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.838+0000 7ff68def8700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 0x7ff67403afa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff670006fd0 tx=0x7ff670006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.838+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 msgr2=0x7ff68819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.838+0000 7ff68def8700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff678006010 tx=0x7ff678004ff0 comp rx=0 tx=0).stop 2024-01-25T15:59:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 shutdown_connections 2024-01-25T15:59:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7ff674038ae0 0x7ff67403afa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 --2- 172.21.15.33:0/3924440449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff688100310 0x7ff68819b620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 >> 172.21.15.33:0/3924440449 conn(0x7ff688076d20 msgr2=0x7ff688101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 shutdown_connections 2024-01-25T15:59:19.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:19.839+0000 7ff68def8700 1 -- 172.21.15.33:0/3924440449 wait complete. 2024-01-25T15:59:20.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch host ls --format=json 2024-01-25T15:59:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:21.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:20 smithi033 ceph-mon[31417]: Added host smithi152 2024-01-25T15:59:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:20 smithi033 ceph-mon[31417]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:22.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:21 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 -- 172.21.15.33:0/3037216238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 msgr2=0x7f78201006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 --2- 172.21.15.33:0/3037216238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f78201006e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f781000b0a0 tx=0x7f781000b3b0 comp rx=0 tx=0).stop 2024-01-25T15:59:22.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 -- 172.21.15.33:0/3037216238 shutdown_connections 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 --2- 172.21.15.33:0/3037216238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f78201006e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 -- 172.21.15.33:0/3037216238 >> 172.21.15.33:0/3037216238 conn(0x7f78200fbb90 msgr2=0x7f78200fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 -- 172.21.15.33:0/3037216238 shutdown_connections 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.176+0000 7f78261a8700 1 -- 172.21.15.33:0/3037216238 wait complete. 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f78261a8700 1 Processor -- start 2024-01-25T15:59:22.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f78261a8700 1 -- start start 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f78261a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f78261a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782019bbc0 con 0x7f7820100300 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f781f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f781f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44774/0 (socket says 172.21.15.33:44774) 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f781f7fe700 1 -- 172.21.15.33:0/3245926286 learned_addr learned my addr 172.21.15.33:0/3245926286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.177+0000 7f781f7fe700 1 -- 172.21.15.33:0/3245926286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7810009d00 con 0x7f7820100300 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f781f7fe700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f78100062a0 tx=0x7f781000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781000e040 con 0x7f7820100300 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7810003c10 con 0x7f7820100300 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782019bdc0 con 0x7f7820100300 2024-01-25T15:59:22.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781001d470 con 0x7f7820100300 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.178+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820195750 con 0x7f7820100300 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f7810019070 con 0x7f7820100300 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782019c1e0 con 0x7f7820100300 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f781cff9700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 0x7f780c03b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f781004cab0 con 0x7f7820100300 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f781effd700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 0x7f780c03b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.179+0000 7f781effd700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 0x7f780c03b040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:22.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.184+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7810026780 con 0x7f7820100300 2024-01-25T15:59:22.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.389+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f782010a3f0 con 0x7f780c038b80 2024-01-25T15:59:22.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.390+0000 7f781cff9700 1 -- 172.21.15.33:0/3245926286 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f782010a3f0 con 0x7f780c038b80 2024-01-25T15:59:22.394 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T15:59:22.394 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.76", "hostname": "smithi076", "labels": [], "status": ""}, {"addr": "172.21.15.152", "hostname": "smithi152", "labels": [], "status": ""}] 2024-01-25T15:59:22.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.392+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 msgr2=0x7f780c03b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.392+0000 7f78261a8700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 0x7f780c03b040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7808006fd0 tx=0x7f7808006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 msgr2=0x7f782019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f78100062a0 tx=0x7f781000bd40 comp rx=0 tx=0).stop 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 shutdown_connections 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f780c038b80 0x7f780c03b040 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 --2- 172.21.15.33:0/3245926286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7820100300 0x7f782019b680 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 >> 172.21.15.33:0/3245926286 conn(0x7f78200fbb90 msgr2=0x7f7820101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 shutdown_connections 2024-01-25T15:59:22.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:22.393+0000 7f78261a8700 1 -- 172.21.15.33:0/3245926286 wait complete. 2024-01-25T15:59:22.990 INFO:tasks.cephadm:Setting crush tunables to default 2024-01-25T15:59:22.990 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd crush tunables default 2024-01-25T15:59:23.014 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:22 smithi033 ceph-mon[31417]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:23.014 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:24.194 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:23 smithi033 ceph-mon[31417]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T15:59:24.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 -- 172.21.15.33:0/2396626294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 msgr2=0x7f8a301006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:24.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/2396626294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a301006f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009910 tx=0x7f8a24009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:24.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 -- 172.21.15.33:0/2396626294 shutdown_connections 2024-01-25T15:59:24.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/2396626294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a301006f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:24.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 -- 172.21.15.33:0/2396626294 >> 172.21.15.33:0/2396626294 conn(0x7f8a300fbb80 msgr2=0x7f8a300fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:24.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 -- 172.21.15.33:0/2396626294 shutdown_connections 2024-01-25T15:59:24.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.707+0000 7f8a36b5b700 1 -- 172.21.15.33:0/2396626294 wait complete. 2024-01-25T15:59:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a36b5b700 1 Processor -- start 2024-01-25T15:59:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a36b5b700 1 -- start start 2024-01-25T15:59:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a36b5b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a36b5b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3006a630 con 0x7f8a30100310 2024-01-25T15:59:24.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a348f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a348f7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44784/0 (socket says 172.21.15.33:44784) 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.708+0000 7f8a348f7700 1 -- 172.21.15.33:0/1128130072 learned_addr learned my addr 172.21.15.33:0/1128130072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a348f7700 1 -- 172.21.15.33:0/1128130072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240095f0 con 0x7f8a30100310 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a348f7700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a24004f40 tx=0x7f8a24005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a2401d070 con 0x7f8a30100310 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a2400bc10 con 0x7f8a30100310 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a240037d0 con 0x7f8a30100310 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a30197ee0 con 0x7f8a30100310 2024-01-25T15:59:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.709+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a30198360 con 0x7f8a30100310 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.710+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f8a2400b6f0 con 0x7f8a30100310 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.710+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30062e10 con 0x7f8a30100310 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.710+0000 7f8a2dffb700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 0x7f8a2003b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.710+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a2404c6d0 con 0x7f8a30100310 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.711+0000 7f8a2ffff700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 0x7f8a2003b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.711+0000 7f8a2ffff700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 0x7f8a2003b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c006fd0 tx=0x7f8a1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:24.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.716+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8a24003a40 con 0x7f8a30100310 2024-01-25T15:59:24.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:24.924+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8a301986b0 con 0x7f8a30100310 2024-01-25T15:59:25.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:24 smithi033 ceph-mon[31417]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.870+0000 7f8a2dffb700 1 -- 172.21.15.33:0/1128130072 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8a24026020 con 0x7f8a30100310 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 msgr2=0x7f8a2003b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 0x7f8a2003b000 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c006fd0 tx=0x7f8a1c006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 msgr2=0x7f8a30197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8a24004f40 tx=0x7f8a24005020 comp rx=0 tx=0).stop 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 shutdown_connections 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8a20038b40 0x7f8a2003b000 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 --2- 172.21.15.33:0/1128130072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a30100310 0x7f8a30197940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 >> 172.21.15.33:0/1128130072 conn(0x7f8a300fbb80 msgr2=0x7f8a30101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 shutdown_connections 2024-01-25T15:59:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:25.873+0000 7f8a36b5b700 1 -- 172.21.15.33:0/1128130072 wait complete. 2024-01-25T15:59:25.877 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-01-25T15:59:26.200 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1128130072' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-01-25T15:59:26.200 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi152", "name": "osd_memory_target"}]: dispatch 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:26.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:26.512 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-01-25T15:59:26.512 INFO:tasks.cephadm:Adding mon.b on smithi076 2024-01-25T15:59:26.513 INFO:tasks.cephadm:Adding mon.c on smithi152 2024-01-25T15:59:26.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi076:172.21.15.76=b;smithi152:172.21.15.152=c' 2024-01-25T15:59:26.736 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:27.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: Updating smithi152:/etc/ceph/ceph.conf 2024-01-25T15:59:27.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1128130072' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:27.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:28.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- 172.21.15.152:0/661791172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 msgr2=0x7f28a8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:28.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 --2- 172.21.15.152:0/661791172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a8101800 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f289c009910 tx=0x7f289c009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- 172.21.15.152:0/661791172 shutdown_connections 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 --2- 172.21.15.152:0/661791172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a8101800 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- 172.21.15.152:0/661791172 >> 172.21.15.152:0/661791172 conn(0x7f28a80fccf0 msgr2=0x7f28a80ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- 172.21.15.152:0/661791172 shutdown_connections 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- 172.21.15.152:0/661791172 wait complete. 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 Processor -- start 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.268+0000 7f28aeb05700 1 -- start start 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28aeb05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:28.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28aeb05700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a819cbf0 con 0x7f28a8101420 2024-01-25T15:59:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28ac8a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28ac8a1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.152:47706/0 (socket says 172.21.15.152:47706) 2024-01-25T15:59:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28ac8a1700 1 -- 172.21.15.152:0/405521008 learned_addr learned my addr 172.21.15.152:0/405521008 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T15:59:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.269+0000 7f28ac8a1700 1 -- 172.21.15.152:0/405521008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c0095f0 con 0x7f28a8101420 2024-01-25T15:59:28.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28ac8a1700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f289c004f40 tx=0x7f289c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f289c01d070 con 0x7f28a8101420 2024-01-25T15:59:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a819cdf0 con 0x7f28a8101420 2024-01-25T15:59:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f289c00bc10 con 0x7f28a8101420 2024-01-25T15:59:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f289c0037d0 con 0x7f28a8101420 2024-01-25T15:59:28.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.270+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a819d210 con 0x7f28a8101420 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.271+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a8062e10 con 0x7f28a8101420 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.271+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f289c00b6f0 con 0x7f28a8101420 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.271+0000 7f28a5ffb700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 0x7f289803afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.271+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f289c04c810 con 0x7f28a8101420 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.271+0000 7f28a7fff700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 0x7f289803afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:28.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.272+0000 7f28a7fff700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 0x7f289803afa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:28.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.276+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f289c003a90 con 0x7f28a8101420 2024-01-25T15:59:28.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:27 smithi033 ceph-mon[31417]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T15:59:28.482 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.478+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi076:172.21.15.76=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f28a810b640 con 0x7f2898038ae0 2024-01-25T15:59:28.488 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.485+0000 7f28a5ffb700 1 -- 172.21.15.152:0/405521008 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f28a810b640 con 0x7f2898038ae0 2024-01-25T15:59:28.488 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mon update... 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 msgr2=0x7f289803afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 0x7f289803afa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2894006fd0 tx=0x7f2894006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 msgr2=0x7f28a819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f289c004f40 tx=0x7f289c005020 comp rx=0 tx=0).stop 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 shutdown_connections 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2898038ae0 0x7f289803afa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 --2- 172.21.15.152:0/405521008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28a8101420 0x7f28a819c6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 >> 172.21.15.152:0/405521008 conn(0x7f28a80fccf0 msgr2=0x7f28a8102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 shutdown_connections 2024-01-25T15:59:28.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:28.488+0000 7f28aeb05700 1 -- 172.21.15.152:0/405521008 wait complete. 2024-01-25T15:59:29.025 DEBUG:teuthology.orchestra.run.smithi076:mon.b> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.b.service 2024-01-25T15:59:29.028 DEBUG:teuthology.orchestra.run.smithi152:mon.c> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.c.service 2024-01-25T15:59:29.030 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-25T15:59:29.030 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph mon dump -f json 2024-01-25T15:59:29.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:29.749 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:30.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:29 smithi033 ceph-mon[31417]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi076:172.21.15.76=b;smithi152:172.21.15.152=c", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:30.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:29 smithi033 ceph-mon[31417]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi076:172.21.15.76=b;smithi152:172.21.15.152=c;count:3 2024-01-25T15:59:30.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:29 smithi033 ceph-mon[31417]: Deploying daemon mon.c on smithi152 2024-01-25T15:59:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:30 smithi033 ceph-mon[31417]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 -- 172.21.15.152:0/3696398033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 msgr2=0x7f33101065a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 --2- 172.21.15.152:0/3696398033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f33101065a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009910 tx=0x7f32f8009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 -- 172.21.15.152:0/3696398033 shutdown_connections 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 --2- 172.21.15.152:0/3696398033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f33101065a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 -- 172.21.15.152:0/3696398033 >> 172.21.15.152:0/3696398033 conn(0x7f33100fb330 msgr2=0x7f33100fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 -- 172.21.15.152:0/3696398033 shutdown_connections 2024-01-25T15:59:32.745 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 -- 172.21.15.152:0/3696398033 wait complete. 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.741+0000 7f331601a700 1 Processor -- start 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f331601a700 1 -- start start 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f331601a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f331601a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331019ed80 con 0x7f33101061c0 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f330f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f330f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.152:47718/0 (socket says 172.21.15.152:47718) 2024-01-25T15:59:32.746 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f330f7fe700 1 -- 172.21.15.152:0/1280688185 learned_addr learned my addr 172.21.15.152:0/1280688185 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T15:59:32.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.742+0000 7f330f7fe700 1 -- 172.21.15.152:0/1280688185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80095f0 con 0x7f33101061c0 2024-01-25T15:59:32.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f330f7fe700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f32f8004f40 tx=0x7f32f8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:32.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32f801d070 con 0x7f33101061c0 2024-01-25T15:59:32.747 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331019efe0 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32f800bbb0 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32f8003810 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.743+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331019f460 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.744+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f32f8003a90 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.744+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3310062e10 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.744+0000 7f330cff9700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 0x7f32fc03afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.744+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f32f804c610 con 0x7f33101061c0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.744+0000 7f330effd700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 0x7f32fc03afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:32.748 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.745+0000 7f330effd700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 0x7f32fc03afa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:32.752 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:32.750+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f32f802ad00 con 0x7f33101061c0 2024-01-25T15:59:33.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.042+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3310199f30 con 0x7f33101061c0 2024-01-25T15:59:33.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.043+0000 7f330cff9700 1 -- 172.21.15.152:0/1280688185 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f32f8026030 con 0x7f33101061c0 2024-01-25T15:59:33.046 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:59:33.046 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":1,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","modified":"2024-01-25T15:57:51.245260Z","created":"2024-01-25T15:57:51.245260Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-01-25T15:59:33.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 msgr2=0x7f32fc03afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:33.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 0x7f32fc03afa0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3300006fd0 tx=0x7f3300006e40 comp rx=0 tx=0).stop 2024-01-25T15:59:33.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 msgr2=0x7f331019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:33.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f32f8004f40 tx=0x7f32f8005020 comp rx=0 tx=0).stop 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 shutdown_connections 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f32fc038ae0 0x7f32fc03afa0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 --2- 172.21.15.152:0/1280688185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33101061c0 0x7f331019e840 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 >> 172.21.15.152:0/1280688185 conn(0x7f33100fb330 msgr2=0x7f33100fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 shutdown_connections 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:33.045+0000 7f331601a700 1 -- 172.21.15.152:0/1280688185 wait complete. 2024-01-25T15:59:33.049 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 1 2024-01-25T15:59:33.232 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:33 smithi152 systemd[1]: Started Ceph mon.c for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T15:59:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:33 smithi033 ceph-mon[31417]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:33.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:33 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/1280688185' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-25T15:59:34.792 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-25T15:59:34.792 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph mon dump -f json 2024-01-25T15:59:35.036 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T15:59:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:59:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: mon.a calling monitor election 2024-01-25T15:59:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: mon.c calling monitor election 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: monmap e2: 2 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: fsmap 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: mgrmap e13: x(active, since 50s) 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: overall HEALTH_OK 2024-01-25T15:59:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:38.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:38.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: mon.a calling monitor election 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: mon.c calling monitor election 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: monmap e2: 2 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: fsmap 2024-01-25T15:59:38.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: mgrmap e13: x(active, since 50s) 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: overall HEALTH_OK 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:38.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:39.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 -- 172.21.15.152:0/931254650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 msgr2=0x7f65d8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 --2- 172.21.15.152:0/931254650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65d8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65ec069aa0 tx=0x7f65e800feb0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 -- 172.21.15.152:0/931254650 shutdown_connections 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 --2- 172.21.15.152:0/931254650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65d8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 -- 172.21.15.152:0/931254650 >> 172.21.15.152:0/931254650 conn(0x7f65ec0fbcf0 msgr2=0x7f65ec0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 -- 172.21.15.152:0/931254650 shutdown_connections 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.514+0000 7f65f37ba700 1 -- 172.21.15.152:0/931254650 wait complete. 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 Processor -- start 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 -- start start 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65ec19d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:39.520 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec19dce0 con 0x7f65d80035c0 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f37ba700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ec19de50 con 0x7f65ec19d750 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f0d55700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f0d55700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:35034/0 (socket says 172.21.15.152:35034) 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f0d55700 1 -- 172.21.15.152:0/2797783959 learned_addr learned my addr 172.21.15.152:0/2797783959 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.515+0000 7f65f1556700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65ec19d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f0d55700 1 -- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 msgr2=0x7f65ec1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f0d55700 1 -- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 msgr2=0x7f65ec1a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f0d55700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f0d55700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f1556700 1 -- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 msgr2=0x7f65ec1a1ae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f1556700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f1556700 1 -- 172.21.15.152:0/2797783959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e800f900 con 0x7f65d80035c0 2024-01-25T15:59:39.521 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f1556700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65ec19d210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65e8010010 tx=0x7f65e8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:39.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f65e8013960 con 0x7f65d80035c0 2024-01-25T15:59:39.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f65e8004020 con 0x7f65d80035c0 2024-01-25T15:59:39.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ec1a2080 con 0x7f65d80035c0 2024-01-25T15:59:39.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.516+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f65e8003b80 con 0x7f65d80035c0 2024-01-25T15:59:39.522 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.517+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ec1a25d0 con 0x7f65d80035c0 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.518+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f65e8011590 con 0x7f65d80035c0 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.518+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ec103f40 con 0x7f65d80035c0 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.518+0000 7f65e27fc700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 0x7f65d003ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.518+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f65e804e480 con 0x7f65d80035c0 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.518+0000 7f65f0d55700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 0x7f65d003ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:39.523 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.519+0000 7f65f0d55700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 0x7f65d003ad40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009180 tx=0x7f65dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:39.526 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.523+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f65e8003e40 con 0x7f65d80035c0 2024-01-25T15:59:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:39 smithi152 ceph-mon[36015]: Deploying daemon mon.b on smithi076 2024-01-25T15:59:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:39.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:39 smithi033 ceph-mon[31417]: Deploying daemon mon.b on smithi076 2024-01-25T15:59:39.717 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:39.800 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.797+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f65ec19e570 con 0x7f65d80035c0 2024-01-25T15:59:39.801 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.798+0000 7f65e27fc700 1 -- 172.21.15.152:0/2797783959 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f65e802b3a0 con 0x7f65d80035c0 2024-01-25T15:59:39.801 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:59:39.801 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":2,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","modified":"2024-01-25T15:59:33.213938Z","created":"2024-01-25T15:57:51.245260Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-01-25T15:59:39.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 msgr2=0x7f65d003ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:39.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 0x7f65d003ad40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009180 tx=0x7f65dc008040 comp rx=0 tx=0).stop 2024-01-25T15:59:39.803 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 msgr2=0x7f65ec19d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65ec19d210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65e8010010 tx=0x7f65e8004f10 comp rx=0 tx=0).stop 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 shutdown_connections 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f65d0038880 0x7f65d003ad40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65d80035c0 0x7f65ec19d210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 --2- 172.21.15.152:0/2797783959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f65ec19d750 0x7f65ec1a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 >> 172.21.15.152:0/2797783959 conn(0x7f65ec0fbcf0 msgr2=0x7f65ec0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 shutdown_connections 2024-01-25T15:59:39.804 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:39.800+0000 7f65f37ba700 1 -- 172.21.15.152:0/2797783959 wait complete. 2024-01-25T15:59:39.805 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 2 2024-01-25T15:59:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:40 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/2797783959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-25T15:59:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:40 smithi152 ceph-mon[36015]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:40.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:40 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/2797783959' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-01-25T15:59:40.730 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:40 smithi033 ceph-mon[31417]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:41.420 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-01-25T15:59:41.420 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph mon dump -f json 2024-01-25T15:59:41.664 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T15:59:42.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:42 smithi076 systemd[1]: Started Ceph mon.b for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T15:59:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: mon.c calling monitor election 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: mon.a calling monitor election 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-25T15:59:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: fsmap 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: mgrmap e13: x(active, since 59s) 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: mon.b (rank 2) addr [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] is down (out of quorum) 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: mon.c calling monitor election 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: mon.a calling monitor election 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: fsmap 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: mgrmap e13: x(active, since 59s) 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-01-25T15:59:48.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: mon.b (rank 2) addr [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] is down (out of quorum) 2024-01-25T15:59:48.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 -- 172.21.15.152:0/773459186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 msgr2=0x7f8e0c0086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/773459186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e0c0086b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8e240689e0 tx=0x7f8e1400fe90 comp rx=0 tx=0).stop 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 -- 172.21.15.152:0/773459186 shutdown_connections 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/773459186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e0c0086b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 -- 172.21.15.152:0/773459186 >> 172.21.15.152:0/773459186 conn(0x7f8e240fb8f0 msgr2=0x7f8e240fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:48.954 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 -- 172.21.15.152:0/773459186 shutdown_connections 2024-01-25T15:59:48.955 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.952+0000 7f8e2aa39700 1 -- 172.21.15.152:0/773459186 wait complete. 2024-01-25T15:59:48.955 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:48.955 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:48 smithi152 ceph-mon[36015]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:48.955 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:48.955 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:48.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 Processor -- start 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 -- start start 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e24196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24197340 0x7f8e2419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e24197a30 con 0x7f8e0c006240 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241a38e0 con 0x7f8e24197340 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e2aa39700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241a3a20 con 0x7f8e2419bb70 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e28fd6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e28fd6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:53834/0 (socket says 172.21.15.152:53834) 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e28fd6700 1 -- 172.21.15.152:0/1426383393 learned_addr learned my addr 172.21.15.152:0/1426383393 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.953+0000 7f8e23fff700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e24196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:48.958 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e237fe700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24197340 0x7f8e2419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 -- 172.21.15.152:0/1426383393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24197340 msgr2=0x7f8e2419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24197340 0x7f8e2419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 -- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 msgr2=0x7f8e24196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e24196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 -- 172.21.15.152:0/1426383393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1400f8e0 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e28fd6700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00ba50 tx=0x7f8e1c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00c9b0 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1c014440 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e241a3d00 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c007590 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.954+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e241a41b0 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.955+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f8e1c0077b0 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.955+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24062e10 con 0x7f8e2419bb70 2024-01-25T15:59:48.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.955+0000 7f8e217fa700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 0x7f8e1003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.955+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8e1c0148d0 con 0x7f8e2419bb70 2024-01-25T15:59:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.955+0000 7f8e23fff700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 0x7f8e1003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:48.960 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.956+0000 7f8e23fff700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 0x7f8e1003b430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005b40 tx=0x7f8e14005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:48.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:48.960+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8e1c01b8b0 con 0x7f8e2419bb70 2024-01-25T15:59:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:48 smithi033 ceph-mon[31417]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:49.241 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.239+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8e24109230 con 0x7f8e2419bb70 2024-01-25T15:59:49.244 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.240+0000 7f8e217fa700 1 -- 172.21.15.152:0/1426383393 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1279 (secure 0 0 0) 0x7f8e1c01b2c0 con 0x7f8e2419bb70 2024-01-25T15:59:49.244 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-25T15:59:49.244 INFO:teuthology.orchestra.run.smithi152.stdout:{"epoch":3,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","modified":"2024-01-25T15:59:42.691551Z","created":"2024-01-25T15:57:51.245260Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:3300","nonce":0},{"type":"v1","addr":"172.21.15.152:6789","nonce":0}]},"addr":"172.21.15.152:6789/0","public_addr":"172.21.15.152:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:3300","nonce":0},{"type":"v1","addr":"172.21.15.76:6789","nonce":0}]},"addr":"172.21.15.76:6789/0","public_addr":"172.21.15.76:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 msgr2=0x7f8e1003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 0x7f8e1003b430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8e14005b40 tx=0x7f8e14005ad0 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 msgr2=0x7f8e2419c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00ba50 tx=0x7f8e1c00bd60 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 shutdown_connections 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e10038f70 0x7f8e1003b430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e2419bb70 0x7f8e2419c020 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c006240 0x7f8e24196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 --2- 172.21.15.152:0/1426383393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24197340 0x7f8e2419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 >> 172.21.15.152:0/1426383393 conn(0x7f8e240fb8f0 msgr2=0x7f8e24104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:49.248 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 shutdown_connections 2024-01-25T15:59:49.249 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:49.242+0000 7f8e2aa39700 1 -- 172.21.15.152:0/1426383393 wait complete. 2024-01-25T15:59:49.249 INFO:teuthology.orchestra.run.smithi152.stderr:dumped monmap epoch 3 2024-01-25T15:59:49.894 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-01-25T15:59:49.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph config generate-minimal-conf 2024-01-25T15:59:50.145 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: mon.c calling monitor election 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: mon.a calling monitor election 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: fsmap 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: mgrmap e13: x(active, since 59s) 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: mon.b (rank 2) addr [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] is down (out of quorum) 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mon.b calling monitor election 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mon.b calling monitor election 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mon.c calling monitor election 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mon.a calling monitor election 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: fsmap 2024-01-25T15:59:50.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: mgrmap e13: x(active, since 61s) 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Cluster is now healthy 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: overall HEALTH_OK 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Reconfiguring mon.a (unknown last config time)... 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: Reconfiguring daemon mon.a on smithi033 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:50.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mon.b calling monitor election 2024-01-25T15:59:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mon.b calling monitor election 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mon.c calling monitor election 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mon.a calling monitor election 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: fsmap 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: mgrmap e13: x(active, since 61s) 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Cluster is now healthy 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: overall HEALTH_OK 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Reconfiguring mon.a (unknown last config time)... 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: Reconfiguring daemon mon.a on smithi033 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mon.b calling monitor election 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mon.b calling monitor election 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mon.c calling monitor election 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mon.a calling monitor election 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: monmap e3: 3 mons at {a=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],b=[v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0],c=[v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: fsmap 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: osdmap e4: 0 total, 0 up, 0 in 2024-01-25T15:59:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: mgrmap e13: x(active, since 61s) 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Cluster is now healthy 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: overall HEALTH_OK 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Reconfiguring mon.a (unknown last config time)... 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: Reconfiguring daemon mon.a on smithi033 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T15:59:51.801 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 15:59:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T15:59:51.691+0000 7f958ba26700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 -- 172.21.15.33:0/1541129408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c100310 msgr2=0x7f785c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 --2- 172.21.15.33:0/1541129408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c100310 0x7f785c1006f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7858009910 tx=0x7f7858009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 -- 172.21.15.33:0/1541129408 shutdown_connections 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 --2- 172.21.15.33:0/1541129408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c100310 0x7f785c1006f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 -- 172.21.15.33:0/1541129408 >> 172.21.15.33:0/1541129408 conn(0x7f785c076d20 msgr2=0x7f785c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:52.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 -- 172.21.15.33:0/1541129408 shutdown_connections 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.286+0000 7f78638cd700 1 -- 172.21.15.33:0/1541129408 wait complete. 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 Processor -- start 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 -- start start 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785c100310 0x7f785c197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 0x7f785c1a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a3600 con 0x7f785c198000 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a3770 con 0x7f785c1a0b30 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f78638cd700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1a3930 con 0x7f785c100310 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44346/0 (socket says 172.21.15.33:44346) 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 -- 172.21.15.33:0/3455869967 learned_addr learned my addr 172.21.15.33:0/3455869967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7861669700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785c100310 0x7f785c197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:52.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7861e6a700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 0x7f785c1a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 -- 172.21.15.33:0/3455869967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785c100310 msgr2=0x7f785c197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785c100310 0x7f785c197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 -- 172.21.15.33:0/3455869967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 msgr2=0x7f785c1a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 0x7f785c1a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7860e68700 1 -- 172.21.15.33:0/3455869967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78580095f0 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.287+0000 7f7861e6a700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 0x7f785c1a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f7860e68700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b780 tx=0x7f784c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c004230 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f784c004390 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c1a3c10 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784c011b30 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.288+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c1a4110 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f784c0036a0 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c1017f0 con 0x7f785c198000 2024-01-25T15:59:52.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f78527fc700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 0x7f784803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:52.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f784c04b0d0 con 0x7f785c198000 2024-01-25T15:59:52.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f7861669700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 0x7f784803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:52.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.289+0000 7f7861669700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 0x7f784803b430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7858006010 tx=0x7f785800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:52.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.294+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f784c01a920 con 0x7f785c198000 2024-01-25T15:59:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.499+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f785c10a4d0 con 0x7f785c198000 2024-01-25T15:59:52.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.499+0000 7f78527fc700 1 -- 172.21.15.33:0/3455869967 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f784c018e40 con 0x7f785c198000 2024-01-25T15:59:52.503 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:59:52.503 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-01-25T15:59:52.503 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T15:59:52.503 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] 2024-01-25T15:59:52.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.501+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 msgr2=0x7f784803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.501+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 0x7f784803b430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7858006010 tx=0x7f785800b540 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.501+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 msgr2=0x7f785c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.501+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f784c00b780 tx=0x7f784c00bb40 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 shutdown_connections 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7848038f70 0x7f784803b430 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f785c1a0b30 0x7f785c1a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f785c100310 0x7f785c197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 --2- 172.21.15.33:0/3455869967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f785c198000 0x7f785c198480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 >> 172.21.15.33:0/3455869967 conn(0x7f785c076d20 msgr2=0x7f785c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 shutdown_connections 2024-01-25T15:59:52.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T15:59:52.502+0000 7f78638cd700 1 -- 172.21.15.33:0/3455869967 wait complete. 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3455869967' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:53.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.317 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-01-25T15:59:53.318 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:59:53.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-01-25T15:59:53.350 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:59:53.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3455869967' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:53.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.418 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:59:53.418 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.conf 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3455869967' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:53.450 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:59:53.450 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:53.518 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:59:53.518 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.conf 2024-01-25T15:59:53.552 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:59:53.552 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-01-25T15:59:53.625 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-01-25T15:59:53.625 INFO:tasks.cephadm:Adding mgr.y on smithi076 2024-01-25T15:59:53.625 INFO:tasks.cephadm:Adding mgr.z on smithi152 2024-01-25T15:59:53.625 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch apply mgr '3;smithi033=x;smithi076=y;smithi152=z' 2024-01-25T15:59:53.898 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T15:59:54.264 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:54 smithi152 ceph-mon[36015]: Reconfiguring mon.b (monmap changed)... 2024-01-25T15:59:54.264 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:54 smithi152 ceph-mon[36015]: Reconfiguring daemon mon.b on smithi076 2024-01-25T15:59:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:54 smithi076 ceph-mon[35642]: Reconfiguring mon.b (monmap changed)... 2024-01-25T15:59:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:54 smithi076 ceph-mon[35642]: Reconfiguring daemon mon.b on smithi076 2024-01-25T15:59:54.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:54 smithi033 ceph-mon[31417]: Reconfiguring mon.b (monmap changed)... 2024-01-25T15:59:54.542 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:54 smithi033 ceph-mon[31417]: Reconfiguring daemon mon.b on smithi076 2024-01-25T15:59:55.289 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:55.290 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.290 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.290 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:55.290 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:55.290 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-01-25T15:59:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-01-25T15:59:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:55.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 -- 172.21.15.152:0/1061005190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 msgr2=0x7fbe30100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:55.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/1061005190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe30100660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009910 tx=0x7fbe20009c20 comp rx=0 tx=0).stop 2024-01-25T15:59:55.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 -- 172.21.15.152:0/1061005190 shutdown_connections 2024-01-25T15:59:55.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/1061005190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe30100660 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.734 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 -- 172.21.15.152:0/1061005190 >> 172.21.15.152:0/1061005190 conn(0x7fbe300fb7c0 msgr2=0x7fbe300fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 -- 172.21.15.152:0/1061005190 shutdown_connections 2024-01-25T15:59:55.735 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.731+0000 7fbe35dfb700 1 -- 172.21.15.152:0/1061005190 wait complete. 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 Processor -- start 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 -- start start 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe3006a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe30199530 0x7fbe3019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:55.737 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3019eae0 con 0x7fbe30100240 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3019ec50 con 0x7fbe30199530 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe35dfb700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3019ee10 con 0x7fbe3019beb0 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:43806/0 (socket says 172.21.15.152:43806) 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe2ffff700 1 -- 172.21.15.152:0/276797561 learned_addr learned my addr 172.21.15.152:0/276797561 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.732+0000 7fbe2f7fe700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe3006a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2effd700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe30199530 0x7fbe3019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 -- 172.21.15.152:0/276797561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe30199530 msgr2=0x7fbe3019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe30199530 0x7fbe3019b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 -- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 msgr2=0x7fbe3006a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe3006a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 -- 172.21.15.152:0/276797561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200095f0 con 0x7fbe3019beb0 2024-01-25T15:59:55.738 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2ffff700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ba50 tx=0x7fbe2800bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800c9b0 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe28014440 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007590 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3019f0f0 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.733+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3019f620 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.734+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7fbe280077b0 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.734+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3018ef10 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.734+0000 7fbe2cff9700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 0x7fbe1803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.734+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbe2804b020 con 0x7fbe3019beb0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.734+0000 7fbe2f7fe700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 0x7fbe1803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T15:59:55.739 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.735+0000 7fbe2f7fe700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 0x7fbe1803b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009fd0 tx=0x7fbe20005200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T15:59:55.742 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.739+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbe2801d400 con 0x7fbe3019beb0 2024-01-25T15:59:55.953 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.950+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi076=y;smithi152=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe3002d540 con 0x7fbe18038f70 2024-01-25T15:59:55.959 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.956+0000 7fbe2cff9700 1 -- 172.21.15.152:0/276797561 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbe3002d540 con 0x7fbe18038f70 2024-01-25T15:59:55.959 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mgr update... 2024-01-25T15:59:55.961 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 msgr2=0x7fbe1803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 0x7fbe1803b430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009fd0 tx=0x7fbe20005200 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 msgr2=0x7fbe3019e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800ba50 tx=0x7fbe2800bd60 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 shutdown_connections 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbe18038f70 0x7fbe1803b430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe3019beb0 0x7fbe3019e340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe30100240 0x7fbe3006a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 --2- 172.21.15.152:0/276797561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe30199530 0x7fbe3019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.958+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 >> 172.21.15.152:0/276797561 conn(0x7fbe300fb7c0 msgr2=0x7fbe300fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.959+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 shutdown_connections 2024-01-25T15:59:55.962 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T15:59:55.959+0000 7fbe35dfb700 1 -- 172.21.15.152:0/276797561 wait complete. 2024-01-25T15:59:56.372 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:56 smithi152 ceph-mon[36015]: Reconfiguring mon.c (monmap changed)... 2024-01-25T15:59:56.373 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:56 smithi152 ceph-mon[36015]: Reconfiguring daemon mon.c on smithi152 2024-01-25T15:59:56.373 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.373 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.373 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:56 smithi076 ceph-mon[35642]: Reconfiguring mon.c (monmap changed)... 2024-01-25T15:59:56.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:56 smithi076 ceph-mon[35642]: Reconfiguring daemon mon.c on smithi152 2024-01-25T15:59:56.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:56 smithi033 ceph-mon[31417]: Reconfiguring mon.c (monmap changed)... 2024-01-25T15:59:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:56 smithi033 ceph-mon[31417]: Reconfiguring daemon mon.c on smithi152 2024-01-25T15:59:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:56.649 DEBUG:teuthology.orchestra.run.smithi076:mgr.y> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.y.service 2024-01-25T15:59:56.652 DEBUG:teuthology.orchestra.run.smithi152:mgr.z> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.z.service 2024-01-25T15:59:56.655 INFO:tasks.cephadm:Deploying OSDs... 2024-01-25T15:59:56.655 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T15:59:56.655 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-01-25T15:59:56.670 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-25T15:59:56.670 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-01-25T15:59:56.725 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-25T15:59:56.725 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:56.725 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-01-25T15:59:56.725 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:56.726 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:56.726 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-01-25 15:59:00.229186328 +0000 2024-01-25T15:59:56.726 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-01-25 15:54:14.050378602 +0000 2024-01-25T15:59:56.726 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-01-25 15:54:14.050378602 +0000 2024-01-25T15:59:56.726 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-01-25 15:54:14.050378602 +0000 2024-01-25T15:59:56.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-25T15:59:56.793 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-01-25T15:59:56.793 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-01-25T15:59:56.793 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000192578 s, 2.7 MB/s 2024-01-25T15:59:56.794 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-25T15:59:56.850 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-01-25T15:59:56.903 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-01-25 15:59:00.259185896 +0000 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-01-25 15:54:14.401373331 +0000 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-01-25 15:54:14.401373331 +0000 2024-01-25T15:59:56.904 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-01-25 15:54:14.401373331 +0000 2024-01-25T15:59:56.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-25T15:59:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-01-25T15:59:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-01-25T15:59:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.00018306 s, 2.8 MB/s 2024-01-25T15:59:56.972 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-25T15:59:57.026 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-01-25 15:59:00.292185422 +0000 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-01-25 15:54:14.756368000 +0000 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-01-25 15:54:14.756368000 +0000 2024-01-25T15:59:57.081 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-01-25 15:54:14.756368000 +0000 2024-01-25T15:59:57.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-25T15:59:57.146 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-01-25T15:59:57.146 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-01-25T15:59:57.146 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000197204 s, 2.6 MB/s 2024-01-25T15:59:57.147 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-25T15:59:57.202 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-01-25T15:59:57.256 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-25T15:59:57.256 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-01-25 15:59:00.321185005 +0000 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-01-25 15:54:15.091362970 +0000 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-01-25 15:54:15.091362970 +0000 2024-01-25T15:59:57.257 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-01-25 15:54:15.091362970 +0000 2024-01-25T15:59:57.257 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-25T15:59:57.321 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-01-25T15:59:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-01-25T15:59:57.322 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000207048 s, 2.5 MB/s 2024-01-25T15:59:57.323 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-25T15:59:57.380 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-01-25T15:59:57.380 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-01-25T15:59:57.398 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-25T15:59:57.398 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-01-25T15:59:57.456 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-01-25 15:59:42.116961219 +0000 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-01-25 15:54:28.699302914 +0000 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-01-25 15:54:28.699302914 +0000 2024-01-25T15:59:57.457 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-01-25 15:54:28.699302914 +0000 2024-01-25T15:59:57.457 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi076=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: Saving service mgr spec with placement smithi033=x;smithi076=y;smithi152=z;count:3 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T15:59:57.497 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T15:59:57.498 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T15:59:57.498 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:57 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.524 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi076=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:57.524 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: Saving service mgr spec with placement smithi033=x;smithi076=y;smithi152=z;count:3 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T15:59:57.525 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:57 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.527 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-01-25T15:59:57.527 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-01-25T15:59:57.527 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00031383 s, 1.6 MB/s 2024-01-25T15:59:57.528 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-25T15:59:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi076=y;smithi152=z", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T15:59:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: Saving service mgr spec with placement smithi033=x;smithi076=y;smithi152=z;count:3 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T15:59:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:57 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T15:59:57.588 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-01-25 15:59:42.116961219 +0000 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-01-25 15:54:29.042291802 +0000 2024-01-25T15:59:57.645 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-01-25 15:54:29.042291802 +0000 2024-01-25T15:59:57.646 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-01-25 15:54:29.042291802 +0000 2024-01-25T15:59:57.646 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-25T15:59:57.715 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-01-25T15:59:57.715 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-01-25T15:59:57.715 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000221504 s, 2.3 MB/s 2024-01-25T15:59:57.716 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-25T15:59:57.775 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-01-25 15:59:42.117961189 +0000 2024-01-25T15:59:57.833 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-01-25 15:54:29.380280852 +0000 2024-01-25T15:59:57.834 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-01-25 15:54:29.380280852 +0000 2024-01-25T15:59:57.834 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-01-25 15:54:29.380280852 +0000 2024-01-25T15:59:57.834 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-25T15:59:57.903 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-01-25T15:59:57.903 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-01-25T15:59:57.903 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000237172 s, 2.2 MB/s 2024-01-25T15:59:57.904 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-25T15:59:57.963 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:58.019 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-01-25 15:59:42.117961189 +0000 2024-01-25T15:59:58.020 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-01-25 15:54:29.727269610 +0000 2024-01-25T15:59:58.020 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-01-25 15:54:29.727269610 +0000 2024-01-25T15:59:58.020 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: 2024-01-25 15:54:29.727269610 +0000 2024-01-25T15:59:58.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-25T15:59:58.088 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-01-25T15:59:58.088 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-01-25T15:59:58.088 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000227622 s, 2.2 MB/s 2024-01-25T15:59:58.090 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-25T15:59:58.147 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-25T15:59:58.147 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-01-25T15:59:58.164 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-25T15:59:58.164 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:58.219 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-25 15:59:32.675202777 +0000 2024-01-25T15:59:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-25 15:54:30.142850975 +0000 2024-01-25T15:59:58.220 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-25 15:54:30.142850975 +0000 2024-01-25T15:59:58.220 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-25 15:54:30.142850975 +0000 2024-01-25T15:59:58.220 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-25T15:59:58.287 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-25T15:59:58.287 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-25T15:59:58.287 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000192748 s, 2.7 MB/s 2024-01-25T15:59:58.288 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-25T15:59:58.345 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-25 15:59:32.675202777 +0000 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-25 15:54:30.477846701 +0000 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-25 15:54:30.477846701 +0000 2024-01-25T15:59:58.399 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-25 15:54:30.477846701 +0000 2024-01-25T15:59:58.399 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-25T15:59:58.426 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:58 smithi152 ceph-mon[36015]: Deploying daemon mgr.z on smithi152 2024-01-25T15:59:58.451 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-25T15:59:58.451 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-25T15:59:58.451 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000222807 s, 2.3 MB/s 2024-01-25T15:59:58.452 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-25T15:59:58.508 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-01-25T15:59:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:58 smithi033 ceph-mon[31417]: Deploying daemon mgr.z on smithi152 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-25 15:59:32.675202777 +0000 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-25 15:54:30.810842452 +0000 2024-01-25T15:59:58.566 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-25 15:54:30.810842452 +0000 2024-01-25T15:59:58.567 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-25 15:54:30.810842452 +0000 2024-01-25T15:59:58.567 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-25T15:59:58.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:58 smithi076 ceph-mon[35642]: Deploying daemon mgr.z on smithi152 2024-01-25T15:59:58.701 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-25T15:59:58.702 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-25T15:59:58.702 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000229909 s, 2.2 MB/s 2024-01-25T15:59:58.703 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-25T15:59:58.719 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-25 15:59:58.629903414 +0000 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-25 15:54:31.158838011 +0000 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-25 15:54:31.158838011 +0000 2024-01-25T15:59:58.777 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-25 15:54:31.158838011 +0000 2024-01-25T15:59:58.777 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-25T15:59:58.895 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-25T15:59:58.895 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-25T15:59:58.895 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000221529 s, 2.3 MB/s 2024-01-25T15:59:58.897 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-25T15:59:58.913 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-01-25T15:59:58.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-25T15:59:59.107 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:58 smithi152 systemd[1]: Starting Ceph mgr.z for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T15:59:59.158 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T15:59:59.269 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 15:59:59 smithi033 ceph-mon[31417]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:59.362 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 15:59:59 smithi152 ceph-mon[36015]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:59.362 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 podman[37870]: 2024-01-25 15:59:59.105438682 +0000 UTC m=+0.105552098 container create 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T15:59:59.363 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 podman[37870]: 2024-01-25 15:59:59.017724485 +0000 UTC m=+0.017837862 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T15:59:59.363 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 podman[37870]: 2024-01-25 15:59:59.333374652 +0000 UTC m=+0.333488040 container init 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T15:59:59.363 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 podman[37870]: 2024-01-25 15:59:59.336036039 +0000 UTC m=+0.336149414 container start 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123) 2024-01-25T15:59:59.363 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 ceph-mgr[37889]: -- 172.21.15.152:0/155586520 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56005431d520 con 0x5600542f2800 2024-01-25T15:59:59.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 15:59:59 smithi076 ceph-mon[35642]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T15:59:59.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 bash[37870]: 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 2024-01-25T15:59:59.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 systemd[1]: Started Ceph mgr.z for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:00:00.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T15:59:59.875+0000 7f3a6fb76200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T16:00:00.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 15:59:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T15:59:59.968+0000 7f3a6fb76200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T16:00:00.788 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:00.608+0000 7f3a6fb76200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T16:00:00.788 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:00.700+0000 7f3a6fb76200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T16:00:00.788 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:00.785+0000 7f3a6fb76200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: Deploying daemon mgr.y on smithi076 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: overall HEALTH_OK 2024-01-25T16:00:00.789 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-mon[36015]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:00.807 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: Deploying daemon mgr.y on smithi076 2024-01-25T16:00:00.808 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: overall HEALTH_OK 2024-01-25T16:00:00.808 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:00 smithi076 ceph-mon[35642]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:00.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: Deploying daemon mgr.y on smithi076 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: overall HEALTH_OK 2024-01-25T16:00:00.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:00 smithi033 ceph-mon[31417]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-25T16:00:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-25T16:00:01.050 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-01-25T16:00:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-01-25T16:00:01.050 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111949 s, 937 MB/s 2024-01-25T16:00:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-01-25T16:00:01.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:00.937+0000 7f3a6fb76200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T16:00:01.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.018+0000 7f3a6fb76200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T16:00:01.463 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.194+0000 7f3a6fb76200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T16:00:01.464 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.375+0000 7f3a6fb76200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T16:00:01.464 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.460+0000 7f3a6fb76200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T16:00:01.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-01-25T16:00:01.883 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.553+0000 7f3a6fb76200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T16:00:01.883 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:01.639+0000 7f3a6fb76200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T16:00:01.906 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:00:02.168 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:01 smithi076 systemd[1]: Started Ceph mgr.y for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:00:02.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:02.461+0000 7ff66ca91200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T16:00:02.919 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:02.555+0000 7ff66ca91200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T16:00:03.013 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:02.735+0000 7f3a6fb76200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T16:00:03.013 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:02.873+0000 7f3a6fb76200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T16:00:03.311 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:03.070+0000 7f3a6fb76200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T16:00:03.311 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:03.308+0000 7f3a6fb76200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T16:00:03.311 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:03.312 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.312 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.312 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.312 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.312 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:03 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:03.330 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:03.331 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.237+0000 7ff66ca91200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T16:00:03.331 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.327+0000 7ff66ca91200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 -- 172.21.15.33:0/380187264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b0101910 msgr2=0x7f09b0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/380187264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b0101910 0x7f09b0101d70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f09a40099c0 tx=0x7f09a4009cd0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 -- 172.21.15.33:0/380187264 shutdown_connections 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/380187264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b01022b0 0x7f09b0108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/380187264 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0100710 0x7f09b0100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/380187264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b0101910 0x7f09b0101d70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 -- 172.21.15.33:0/380187264 >> 172.21.15.33:0/380187264 conn(0x7f09b00fbcb0 msgr2=0x7f09b00fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 -- 172.21.15.33:0/380187264 shutdown_connections 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.385+0000 7f09b5aaf700 1 -- 172.21.15.33:0/380187264 wait complete. 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 Processor -- start 2024-01-25T16:00:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 -- start start 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 0x7f09b0104650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0101910 0x7f09b0102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b01039d0 con 0x7f09b01022b0 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0103b40 con 0x7f09b0100710 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09b5aaf700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b01a1780 con 0x7f09b0101910 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09af7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 0x7f09b0104650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42638/0 (socket says 172.21.15.33:42638) 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 -- 172.21.15.33:0/1235954630 learned_addr learned my addr 172.21.15.33:0/1235954630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09af7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 0x7f09b0104650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:37430/0 (socket says 172.21.15.33:37430) 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09aeffd700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0101910 0x7f09b0102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:03.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 -- 172.21.15.33:0/1235954630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0101910 msgr2=0x7f09b0102cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0101910 0x7f09b0102cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 -- 172.21.15.33:0/1235954630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 msgr2=0x7f09b0104650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 0x7f09b0104650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.386+0000 7f09affff700 1 -- 172.21.15.33:0/1235954630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a40095f0 con 0x7f09b01022b0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09affff700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a8007ee0 tx=0x7f09a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8017070 con 0x7f09b01022b0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09a800f040 con 0x7f09b01022b0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b01a1a10 con 0x7f09b01022b0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8005580 con 0x7f09b01022b0 2024-01-25T16:00:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.387+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b01a1f10 con 0x7f09b01022b0 2024-01-25T16:00:03.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 45697+0+0 (secure 0 0 0) 0x7f09a8005780 con 0x7f09b01022b0 2024-01-25T16:00:03.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09acff9700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 0x7f099803b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:03.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f09a804ad70 con 0x7f09b01022b0 2024-01-25T16:00:03.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09af7fe700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 0x7f099803b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b00684a0 con 0x7f09b01022b0 2024-01-25T16:00:03.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.388+0000 7f09af7fe700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 0x7f099803b1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09b0101770 tx=0x7f09a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:03.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.393+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f09a80133f0 con 0x7f09b01022b0 2024-01-25T16:00:03.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:03.604+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f09b0069ba0 con 0x7f0998038d20 2024-01-25T16:00:03.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:03.495+0000 7f3a6fb76200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:00:03.662 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.412+0000 7ff66ca91200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T16:00:03.662 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.574+0000 7ff66ca91200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T16:00:03.663 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.659+0000 7ff66ca91200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T16:00:03.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:03.835+0000 7ff66ca91200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T16:00:03.948 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:03.681+0000 7f3a6fb76200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T16:00:04.285 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:04.016+0000 7ff66ca91200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:04.101+0000 7ff66ca91200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:04.195+0000 7ff66ca91200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:04.282+0000 7ff66ca91200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:04.286 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:04.287 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:04.305 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:04.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:03.944+0000 7f3a6fb76200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:04.030+0000 7f3a6fb76200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-01-25T16:00:04.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:04.883 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:04.452+0000 7f3a6fb76200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T16:00:05.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:05 smithi033 ceph-mon[31417]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:05.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:05 smithi033 ceph-mon[31417]: Reconfiguring mgr.x (unknown last config time)... 2024-01-25T16:00:05.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:05 smithi033 ceph-mon[31417]: Reconfiguring daemon mgr.x on smithi033 2024-01-25T16:00:05.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:05 smithi033 ceph-mon[31417]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:05.351 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-mon[35642]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:05.351 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-mon[35642]: Reconfiguring mgr.x (unknown last config time)... 2024-01-25T16:00:05.352 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-mon[35642]: Reconfiguring daemon mgr.x on smithi033 2024-01-25T16:00:05.352 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-mon[35642]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:05.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:05.237+0000 7f3a6fb76200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T16:00:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-mon[36015]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-mon[36015]: Reconfiguring mgr.x (unknown last config time)... 2024-01-25T16:00:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-mon[36015]: Reconfiguring daemon mgr.x on smithi033 2024-01-25T16:00:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-mon[36015]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:05.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:05.348+0000 7ff66ca91200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T16:00:05.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:05.479+0000 7ff66ca91200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T16:00:06.077 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:05.669+0000 7ff66ca91200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T16:00:06.077 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:05.897+0000 7ff66ca91200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T16:00:06.077 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:06.074+0000 7ff66ca91200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:00:06.098 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:05.804+0000 7f3a6fb76200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T16:00:06.099 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:05.899+0000 7f3a6fb76200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T16:00:06.099 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:00:06.095+0000 7f3a6fb76200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T16:00:06.418 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:06.260+0000 7ff66ca91200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T16:00:06.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:06.525+0000 7ff66ca91200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:00:06.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:06.611+0000 7ff66ca91200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T16:00:07.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:07.019+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) v1 ==== 90768+0+0 (secure 0 0 0) 0x7f09a8012350 con 0x7f09b01022b0 2024-01-25T16:00:07.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: Standby manager daemon z started 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:00:07.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:06 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:07.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: Standby manager daemon z started 2024-01-25T16:00:07.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:00:07.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:07.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:00:07.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:06 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:07.418 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:07.004+0000 7ff66ca91200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: Standby manager daemon z started 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:00:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:06 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/2185603115' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:08.168 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:07.795+0000 7ff66ca91200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: mgrmap e14: x(active, since 79s), standbys: z 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]: dispatch 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]': finished 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: osdmap e5: 1 total, 0 up, 1 in 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:08.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3175545884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:08.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: mgrmap e14: x(active, since 79s), standbys: z 2024-01-25T16:00:08.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:00:08.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]: dispatch 2024-01-25T16:00:08.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]': finished 2024-01-25T16:00:08.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: osdmap e5: 1 total, 0 up, 1 in 2024-01-25T16:00:08.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:08.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:08 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3175545884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: mgrmap e14: x(active, since 79s), standbys: z 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]: dispatch 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3016749135' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83211537-b4b8-4002-921b-1a85501279cb"}]': finished 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: osdmap e5: 1 total, 0 up, 1 in 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:08.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:08 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3175545884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:08.653 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:08.358+0000 7ff66ca91200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T16:00:08.653 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:08.454+0000 7ff66ca91200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T16:00:08.653 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:00:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:00:08.650+0000 7ff66ca91200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T16:00:09.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:09.030+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7f09a8020160 con 0x7f09b01022b0 2024-01-25T16:00:09.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: Standby manager daemon y started 2024-01-25T16:00:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:00:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:00:09.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:09 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: Standby manager daemon y started 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:00:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:09 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:09.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:09.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: Standby manager daemon y started 2024-01-25T16:00:09.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:00:09.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:00:09.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:00:09.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:09 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/36887647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:00:10.295 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:10 smithi076 ceph-mon[35642]: mgrmap e15: x(active, since 81s), standbys: y, z 2024-01-25T16:00:10.295 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:10 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:00:10.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:10 smithi033 ceph-mon[31417]: mgrmap e15: x(active, since 81s), standbys: y, z 2024-01-25T16:00:10.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:00:10.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:10 smithi152 ceph-mon[36015]: mgrmap e15: x(active, since 81s), standbys: y, z 2024-01-25T16:00:10.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:10 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:00:11.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:11 smithi076 ceph-mon[35642]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:11 smithi033 ceph-mon[31417]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:11 smithi152 ceph-mon[36015]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:13.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:13 smithi033 ceph-mon[31417]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:13.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:13 smithi152 ceph-mon[36015]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:13 smithi076 ceph-mon[35642]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:15.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:15 smithi033 ceph-mon[31417]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:15.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-25T16:00:15.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:15 smithi152 ceph-mon[36015]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-25T16:00:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:15 smithi076 ceph-mon[35642]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-01-25T16:00:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:16.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:16 smithi033 ceph-mon[31417]: Deploying daemon osd.0 on smithi033 2024-01-25T16:00:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:16 smithi152 ceph-mon[36015]: Deploying daemon osd.0 on smithi033 2024-01-25T16:00:16.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:16 smithi076 ceph-mon[35642]: Deploying daemon osd.0 on smithi033 2024-01-25T16:00:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:17 smithi152 ceph-mon[36015]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:17 smithi076 ceph-mon[35642]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:17.456 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:17 smithi033 ceph-mon[31417]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:19.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:19 smithi033 ceph-mon[31417]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:19 smithi152 ceph-mon[36015]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:19 smithi076 ceph-mon[35642]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:21.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:21.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:20 smithi152 ceph-mon[36015]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:21.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:21.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:20 smithi076 ceph-mon[35642]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:21.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:21.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:21.223 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:20 smithi033 ceph-mon[31417]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:23.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:23.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:23.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:22 smithi033 ceph-mon[31417]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:22 smithi152 ceph-mon[36015]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:23.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:23.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:23.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:23.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:22 smithi076 ceph-mon[35642]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:23.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.623+0000 7f09acff9700 1 -- 172.21.15.33:0/1235954630 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f09b0069ba0 con 0x7f0998038d20 2024-01-25T16:00:23.627 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 msgr2=0x7f099803b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 0x7f099803b1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09b0101770 tx=0x7f09a000b410 comp rx=0 tx=0).stop 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 msgr2=0x7f09b0103230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f09a8007ee0 tx=0x7f09a800d3b0 comp rx=0 tx=0).stop 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 shutdown_connections 2024-01-25T16:00:23.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0998038d20 0x7f099803b1e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09b01022b0 0x7f09b0103230 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09b0100710 0x7f09b0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 --2- 172.21.15.33:0/1235954630 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09b0101910 0x7f09b0102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 >> 172.21.15.33:0/1235954630 conn(0x7f09b00fbcb0 msgr2=0x7f09b0069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 shutdown_connections 2024-01-25T16:00:23.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:00:23.625+0000 7f09b5aaf700 1 -- 172.21.15.33:0/1235954630 wait complete. 2024-01-25T16:00:24.225 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.0.service 2024-01-25T16:00:24.227 INFO:tasks.cephadm:Deploying osd.1 on smithi076 with /dev/vg_nvme/lv_4... 2024-01-25T16:00:24.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-25T16:00:24.466 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:00:24.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:24.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:24 smithi033 ceph-mon[31417]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-25T16:00:24.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:24 smithi033 ceph-mon[31417]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:24 smithi152 ceph-mon[36015]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-25T16:00:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:24 smithi152 ceph-mon[36015]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:24.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:24.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:24.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:24 smithi076 ceph-mon[35642]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-01-25T16:00:24.902 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:24 smithi076 ceph-mon[35642]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:25.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:25 smithi033 ceph-mon[31417]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-25T16:00:25.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:25 smithi033 ceph-mon[31417]: osdmap e6: 1 total, 0 up, 1 in 2024-01-25T16:00:25.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:25 smithi033 ceph-mon[31417]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:00:25.738 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:25.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:25 smithi076 ceph-mon[35642]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-25T16:00:25.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:25 smithi076 ceph-mon[35642]: osdmap e6: 1 total, 0 up, 1 in 2024-01-25T16:00:25.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:25 smithi076 ceph-mon[35642]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:00:25.841 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:25 smithi152 ceph-mon[36015]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-01-25T16:00:26.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:25 smithi152 ceph-mon[36015]: osdmap e6: 1 total, 0 up, 1 in 2024-01-25T16:00:26.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:25 smithi152 ceph-mon[36015]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:00:26.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013102 s, 800 MB/s 2024-01-25T16:00:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: osdmap e7: 1 total, 0 up, 1 in 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' 2024-01-25T16:00:26.728 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: osdmap e7: 1 total, 0 up, 1 in 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' 2024-01-25T16:00:26.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:26.815 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0[46056]: 2024-01-25T16:00:26.464+0000 7f22a7474700 -1 osd.0 0 waiting for initial osdmap 2024-01-25T16:00:26.815 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:00:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0[46056]: 2024-01-25T16:00:26.470+0000 7f229e264700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:00:27.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_4 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: osdmap e7: 1 total, 0 up, 1 in 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: from='osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098]' entity='osd.0' 2024-01-25T16:00:27.034 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:27.272 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi033 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 17093M 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:27.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:27.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] boot 2024-01-25T16:00:27.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: osdmap e8: 1 total, 1 up, 1 in 2024-01-25T16:00:27.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi033 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 17093M 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] boot 2024-01-25T16:00:28.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: osdmap e8: 1 total, 1 up, 1 in 2024-01-25T16:00:28.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi033 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:28.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 17093M 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: osd.0 [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] boot 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: osdmap e8: 1 total, 1 up, 1 in 2024-01-25T16:00:28.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:00:28.747 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:28 smithi076 ceph-mon[35642]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:28.747 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:28 smithi076 ceph-mon[35642]: osdmap e9: 1 total, 1 up, 1 in 2024-01-25T16:00:28.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- 172.21.15.76:0/653601038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c101810 msgr2=0x7efc2c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:28.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 --2- 172.21.15.76:0/653601038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c101810 0x7efc2c101c30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc280099c0 tx=0x7efc28009cd0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- 172.21.15.76:0/653601038 shutdown_connections 2024-01-25T16:00:28.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 --2- 172.21.15.76:0/653601038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c1033b0 0x7efc2c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 --2- 172.21.15.76:0/653601038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c101810 0x7efc2c101c30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 --2- 172.21.15.76:0/653601038 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c102a10 0x7efc2c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- 172.21.15.76:0/653601038 >> 172.21.15.76:0/653601038 conn(0x7efc2c0fcdf0 msgr2=0x7efc2c0ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- 172.21.15.76:0/653601038 shutdown_connections 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- 172.21.15.76:0/653601038 wait complete. 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 Processor -- start 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.744+0000 7efc33b55700 1 -- start start 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 0x7efc2c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c102a10 0x7efc2c197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c19f2a0 con 0x7efc2c102a10 2024-01-25T16:00:28.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c19f3e0 con 0x7efc2c1033b0 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c19f570 con 0x7efc2c101810 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc318f1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 0x7efc2c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59072/0 (socket says 172.21.15.76:59072) 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 -- 172.21.15.76:0/2420152970 learned_addr learned my addr 172.21.15.76:0/2420152970 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc318f1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 0x7efc2c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.76:58560/0 (socket says 172.21.15.76:58560) 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc310f0700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c102a10 0x7efc2c197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:28.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 -- 172.21.15.76:0/2420152970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 msgr2=0x7efc2c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 0x7efc2c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 -- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c102a10 msgr2=0x7efc2c197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c102a10 0x7efc2c197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 -- 172.21.15.76:0/2420152970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc280095f0 con 0x7efc2c1033b0 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc320f2700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ada0 tx=0x7efc2400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2400f070 con 0x7efc2c1033b0 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc24007b30 con 0x7efc2c1033b0 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc240053e0 con 0x7efc2c1033b0 2024-01-25T16:00:28.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.745+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c19f850 con 0x7efc2c1033b0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.746+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c19fd00 con 0x7efc2c1033b0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.746+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c108cf0 con 0x7efc2c1033b0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.747+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7efc2400c980 con 0x7efc2c1033b0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.748+0000 7efc22ffd700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 0x7efc180a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.748+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7efc240cacd0 con 0x7efc2c1033b0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.748+0000 7efc318f1700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 0x7efc180a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.749+0000 7efc318f1700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 0x7efc180a37a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc28003820 tx=0x7efc28005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:28.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.752+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7efc24095c50 con 0x7efc2c1033b0 2024-01-25T16:00:28.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:28.951+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efc2c02d540 con 0x7efc180a12e0 2024-01-25T16:00:29.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:28 smithi033 ceph-mon[31417]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:29.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:28 smithi033 ceph-mon[31417]: osdmap e9: 1 total, 1 up, 1 in 2024-01-25T16:00:29.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:28 smithi152 ceph-mon[36015]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:29.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:28 smithi152 ceph-mon[36015]: osdmap e9: 1 total, 1 up, 1 in 2024-01-25T16:00:29.805 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:29 smithi076 ceph-mon[35642]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:29.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:29.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:29.806 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:30.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:29 smithi033 ceph-mon[31417]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:30.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:30.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:30.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:30.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:29 smithi152 ceph-mon[36015]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:30.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:30.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:30.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:30.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:30 smithi076 ceph-mon[35642]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:31.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:30 smithi152 ceph-mon[36015]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:30 smithi033 ceph-mon[31417]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:32.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:31 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/3038908259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:31 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:31 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]': finished 2024-01-25T16:00:32.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:31 smithi152 ceph-mon[36015]: osdmap e10: 2 total, 1 up, 2 in 2024-01-25T16:00:32.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:32.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:31 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/3038908259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:31 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:31 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]': finished 2024-01-25T16:00:32.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:31 smithi076 ceph-mon[35642]: osdmap e10: 2 total, 1 up, 2 in 2024-01-25T16:00:32.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:32.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:31 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/3038908259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:31 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]: dispatch 2024-01-25T16:00:32.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:31 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c90ee0f1-8aa4-48ce-8068-df0d16d4c86f"}]': finished 2024-01-25T16:00:32.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:31 smithi033 ceph-mon[31417]: osdmap e10: 2 total, 1 up, 2 in 2024-01-25T16:00:32.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:33.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:32 smithi152 ceph-mon[36015]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:33.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:32 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/1259283351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:33.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:32 smithi076 ceph-mon[35642]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:33.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:32 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/1259283351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:32 smithi033 ceph-mon[31417]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:32 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/1259283351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:35.362 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:35 smithi076 ceph-mon[35642]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:35.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:35 smithi152 ceph-mon[36015]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:35 smithi033 ceph-mon[31417]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:37.380 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:37 smithi076 ceph-mon[35642]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:37.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:37 smithi152 ceph-mon[36015]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:37 smithi033 ceph-mon[31417]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:39.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:39 smithi033 ceph-mon[31417]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:39.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-25T16:00:39.367 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:39 smithi152 ceph-mon[36015]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-25T16:00:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:39.397 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:39 smithi076 ceph-mon[35642]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:39.397 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-01-25T16:00:39.397 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:40 smithi152 ceph-mon[36015]: Deploying daemon osd.1 on smithi076 2024-01-25T16:00:40.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:40 smithi076 ceph-mon[35642]: Deploying daemon osd.1 on smithi076 2024-01-25T16:00:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:40 smithi033 ceph-mon[31417]: Deploying daemon osd.1 on smithi076 2024-01-25T16:00:41.364 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:41 smithi076 ceph-mon[35642]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:41 smithi152 ceph-mon[36015]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:41 smithi033 ceph-mon[31417]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:43.339 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:43 smithi076 ceph-mon[35642]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:43 smithi152 ceph-mon[36015]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:43 smithi033 ceph-mon[31417]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:45.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:45.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:44 smithi076 ceph-mon[35642]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:45.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:45.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:44 smithi033 ceph-mon[31417]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:45.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:45.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:45.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:44 smithi152 ceph-mon[36015]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:47.165 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:47 smithi076 ceph-mon[35642]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:47.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.213+0000 7efc22ffd700 1 -- 172.21.15.76:0/2420152970 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efc2c02d540 con 0x7efc180a12e0 2024-01-25T16:00:47.217 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 1 on host 'smithi076' 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.215+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 msgr2=0x7efc180a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.215+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 0x7efc180a37a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc28003820 tx=0x7efc28005920 comp rx=0 tx=0).stop 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 msgr2=0x7efc2c19ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ada0 tx=0x7efc2400c330 comp rx=0 tx=0).stop 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 shutdown_connections 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efc180a12e0 0x7efc180a37a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efc2c1033b0 0x7efc2c19ecd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc2c101810 0x7efc2c197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:47.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 --2- 172.21.15.76:0/2420152970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c102a10 0x7efc2c197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 >> 172.21.15.76:0/2420152970 conn(0x7efc2c0fcdf0 msgr2=0x7efc2c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:00:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 shutdown_connections 2024-01-25T16:00:47.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:00:47.216+0000 7efc33b55700 1 -- 172.21.15.76:0/2420152970 wait complete. 2024-01-25T16:00:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:47 smithi152 ceph-mon[36015]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:47 smithi033 ceph-mon[31417]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:47.811 DEBUG:teuthology.orchestra.run.smithi076:osd.1> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.1.service 2024-01-25T16:00:47.814 INFO:tasks.cephadm:Deploying osd.2 on smithi152 with /dev/vg_nvme/lv_4... 2024-01-25T16:00:47.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-01-25T16:00:48.059 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.504 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.520 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-01-25T16:00:48.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:48.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: osdmap e11: 2 total, 1 up, 2 in 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:49 smithi152 ceph-mon[36015]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-25T16:00:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: osdmap e11: 2 total, 1 up, 2 in 2024-01-25T16:00:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:49 smithi076 ceph-mon[35642]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:49.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-01-25T16:00:49.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: osdmap e11: 2 total, 1 up, 2 in 2024-01-25T16:00:49.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:49.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: from='osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:49 smithi033 ceph-mon[31417]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114655 s, 915 MB/s 2024-01-25T16:00:49.936 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-01-25T16:00:50.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1[40254]: 2024-01-25T16:00:50.046+0000 7f56050a8700 -1 osd.1 0 waiting for initial osdmap 2024-01-25T16:00:50.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1[40254]: 2024-01-25T16:00:50.054+0000 7f55fd69b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: osdmap e12: 2 total, 1 up, 2 in 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:50.472 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:50 smithi152 ceph-mon[36015]: from='osd.1 ' entity='osd.1' 2024-01-25T16:00:50.473 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_4 2024-01-25T16:00:50.559 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:00:50.560 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: osdmap e12: 2 total, 1 up, 2 in 2024-01-25T16:00:50.560 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.560 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.560 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:50.560 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:50 smithi076 ceph-mon[35642]: from='osd.1 ' entity='osd.1' 2024-01-25T16:00:50.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:00:50.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: osdmap e12: 2 total, 1 up, 2 in 2024-01-25T16:00:50.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:50.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-01-25T16:00:50.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:50 smithi033 ceph-mon[31417]: from='osd.1 ' entity='osd.1' 2024-01-25T16:00:50.711 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] boot 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: osdmap e13: 2 total, 2 up, 2 in 2024-01-25T16:00:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:00:51.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:00:51.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:51.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] boot 2024-01-25T16:00:51.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: osdmap e13: 2 total, 2 up, 2 in 2024-01-25T16:00:51.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:51.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:00:51.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:00:51.670 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:51.670 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: osd.1 [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] boot 2024-01-25T16:00:51.670 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: osdmap e13: 2 total, 2 up, 2 in 2024-01-25T16:00:51.670 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:00:52.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 -- 172.21.15.152:0/1228920977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 msgr2=0x7fa0fc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:52.160 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 --2- 172.21.15.152:0/1228920977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc101d70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009990 tx=0x7fa0e8009ca0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 -- 172.21.15.152:0/1228920977 shutdown_connections 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 --2- 172.21.15.152:0/1228920977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 --2- 172.21.15.152:0/1228920977 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc100710 0x7fa0fc100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 --2- 172.21.15.152:0/1228920977 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc101d70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 -- 172.21.15.152:0/1228920977 >> 172.21.15.152:0/1228920977 conn(0x7fa0fc0fbcb0 msgr2=0x7fa0fc0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 -- 172.21.15.152:0/1228920977 shutdown_connections 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.155+0000 7fa102b1b700 1 -- 172.21.15.152:0/1228920977 wait complete. 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 Processor -- start 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 -- start start 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0fc100710 0x7fa0fc197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19f2c0 con 0x7fa0fc100710 2024-01-25T16:00:52.161 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19f400 con 0x7fa0fc1022b0 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa102b1b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc19f590 con 0x7fa0fc101910 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:57684/0 (socket says 172.21.15.152:57684) 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 -- 172.21.15.152:0/2721096671 learned_addr learned my addr 172.21.15.152:0/2721096671 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 -- 172.21.15.152:0/2721096671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 msgr2=0x7fa0fc19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa1010b8700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 -- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0fc100710 msgr2=0x7fa0fc197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0fc100710 0x7fa0fc197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 -- 172.21.15.152:0/2721096671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80095f0 con 0x7fa0fc101910 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa1010b8700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.156+0000 7fa0fbfff700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b5c0 tx=0x7fa0e800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.157+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e801d070 con 0x7fa0fc101910 2024-01-25T16:00:52.162 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.157+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e8003960 con 0x7fa0fc101910 2024-01-25T16:00:52.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.157+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc19f810 con 0x7fa0fc101910 2024-01-25T16:00:52.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.157+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80056a0 con 0x7fa0fc101910 2024-01-25T16:00:52.163 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.157+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc19fc20 con 0x7fa0fc101910 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.158+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc0684a0 con 0x7fa0fc101910 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.159+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7fa0e8003ad0 con 0x7fa0fc101910 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.159+0000 7fa0f9ffb700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 0x7fa0ec0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.160+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fa0e80cc4f0 con 0x7fa0fc101910 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.160+0000 7fa1008b7700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 0x7fa0ec0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:00:52.164 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.160+0000 7fa1008b7700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 0x7fa0ec0a37a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc101770 tx=0x7fa0f0005ca0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:00:52.166 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.163+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa0e8097250 con 0x7fa0fc101910 2024-01-25T16:00:52.367 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:00:52.364+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0fc069ba0 con 0x7fa0ec0a12e0 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi076 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 17080M 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.369 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:52.369 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.369 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.369 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.369 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:52 smithi152 ceph-mon[36015]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:52.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi076 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 17080M 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:52 smithi033 ceph-mon[31417]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:52.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi076 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 17080M 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:00:52.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:52 smithi076 ceph-mon[35642]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:53 smithi033 ceph-mon[31417]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:53 smithi033 ceph-mon[31417]: osdmap e14: 2 total, 2 up, 2 in 2024-01-25T16:00:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:53 smithi152 ceph-mon[36015]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:53 smithi152 ceph-mon[36015]: osdmap e14: 2 total, 2 up, 2 in 2024-01-25T16:00:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:53 smithi076 ceph-mon[35642]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:00:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:00:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:00:53.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:00:53.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:53 smithi076 ceph-mon[35642]: osdmap e14: 2 total, 2 up, 2 in 2024-01-25T16:00:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:54 smithi033 ceph-mon[31417]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:54.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:54 smithi152 ceph-mon[36015]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:54.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:54 smithi076 ceph-mon[35642]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:55 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/3210293154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:55 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:55 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]': finished 2024-01-25T16:00:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:55 smithi152 ceph-mon[36015]: osdmap e15: 3 total, 2 up, 3 in 2024-01-25T16:00:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:00:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:55 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/3210293154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:55 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:55 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]': finished 2024-01-25T16:00:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:55 smithi033 ceph-mon[31417]: osdmap e15: 3 total, 2 up, 3 in 2024-01-25T16:00:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:00:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:55 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/3210293154' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:55 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]: dispatch 2024-01-25T16:00:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:55 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "337d041f-de02-49ec-87c1-d9e211edacff"}]': finished 2024-01-25T16:00:55.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:55 smithi076 ceph-mon[35642]: osdmap e15: 3 total, 2 up, 3 in 2024-01-25T16:00:55.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:00:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:56 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/3921510051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:56.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:56 smithi033 ceph-mon[31417]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:56 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/3921510051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:56 smithi152 ceph-mon[36015]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:56 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/3921510051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:00:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:56 smithi076 ceph-mon[35642]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:59.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:00:59 smithi152 ceph-mon[36015]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:59.372 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:00:59 smithi076 ceph-mon[35642]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:00:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:00:59 smithi033 ceph-mon[31417]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:00 smithi152 ceph-mon[36015]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:01.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:00 smithi076 ceph-mon[35642]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:00 smithi033 ceph-mon[31417]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:02.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-25T16:01:02.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:02.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-25T16:01:02.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:02.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-01-25T16:01:02.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:02.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:02 smithi152 ceph-mon[36015]: Deploying daemon osd.2 on smithi152 2024-01-25T16:01:02.983 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:02 smithi152 ceph-mon[36015]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:03.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:02 smithi076 ceph-mon[35642]: Deploying daemon osd.2 on smithi152 2024-01-25T16:01:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:02 smithi076 ceph-mon[35642]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:03.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:02 smithi033 ceph-mon[31417]: Deploying daemon osd.2 on smithi152 2024-01-25T16:01:03.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:02 smithi033 ceph-mon[31417]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:05.370 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:05 smithi152 ceph-mon[36015]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:05 smithi076 ceph-mon[35642]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:05 smithi033 ceph-mon[31417]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:07 smithi152 ceph-mon[36015]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:07 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:07 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:07 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:07.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:07 smithi076 ceph-mon[35642]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:07.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:07 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:07 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:07 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:07 smithi033 ceph-mon[31417]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:07 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:07 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:07 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:09.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:09 smithi033 ceph-mon[31417]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:09 smithi152 ceph-mon[36015]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:09.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:09 smithi076 ceph-mon[35642]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:09.832 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 2 on host 'smithi152' 2024-01-25T16:01:09.832 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.828+0000 7fa0f9ffb700 1 -- 172.21.15.152:0/2721096671 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa0fc069ba0 con 0x7fa0ec0a12e0 2024-01-25T16:01:09.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 msgr2=0x7fa0ec0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:09.833 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 0x7fa0ec0a37a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc101770 tx=0x7fa0f0005ca0 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 msgr2=0x7fa0fc197c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa0e800b5c0 tx=0x7fa0e800f690 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 shutdown_connections 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0ec0a12e0 0x7fa0ec0a37a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0fc1022b0 0x7fa0fc19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0fc100710 0x7fa0fc197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 --2- 172.21.15.152:0/2721096671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0fc101910 0x7fa0fc197c70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 >> 172.21.15.152:0/2721096671 conn(0x7fa0fc0fbcb0 msgr2=0x7fa0fc069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 shutdown_connections 2024-01-25T16:01:09.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:01:09.831+0000 7fa102b1b700 1 -- 172.21.15.152:0/2721096671 wait complete. 2024-01-25T16:01:10.374 DEBUG:teuthology.orchestra.run.smithi152:osd.2> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.2.service 2024-01-25T16:01:10.376 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-01-25T16:01:10.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-25T16:01:10.628 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:10.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:10 smithi076 ceph-mon[35642]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:11.042 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-mon[36015]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:11.042 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:01:10.738+0000 7f5619e7e7c0 -1 osd.2 0 log_to_monitors true 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:11.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:10 smithi033 ceph-mon[31417]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-01-25T16:01:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-25T16:01:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-25T16:01:12.596 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-01-25T16:01:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-01-25T16:01:12.596 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110705 s, 947 MB/s 2024-01-25T16:01:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: osdmap e16: 3 total, 2 up, 3 in 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.596 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:12 smithi033 ceph-mon[31417]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: osdmap e16: 3 total, 2 up, 3 in 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:12 smithi152 ceph-mon[36015]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:12.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-01-25T16:01:12.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: osdmap e16: 3 total, 2 up, 3 in 2024-01-25T16:01:12.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:12.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: from='osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:01:12.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:12 smithi076 ceph-mon[35642]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:13.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-01-25T16:01:13.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:01:13.308+0000 7f560ece3700 -1 osd.2 0 waiting for initial osdmap 2024-01-25T16:01:13.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:01:13.314+0000 7f56072d6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:01:13.411 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:01:13.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-mon[36015]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:01:13.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-mon[36015]: osdmap e17: 3 total, 2 up, 3 in 2024-01-25T16:01:13.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.737 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:13 smithi152 ceph-mon[36015]: from='osd.2 ' entity='osd.2' 2024-01-25T16:01:13.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:13 smithi033 ceph-mon[31417]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:01:13.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:13 smithi033 ceph-mon[31417]: osdmap e17: 3 total, 2 up, 3 in 2024-01-25T16:01:13.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:13 smithi033 ceph-mon[31417]: from='osd.2 ' entity='osd.2' 2024-01-25T16:01:13.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:13 smithi076 ceph-mon[35642]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:01:13.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:13 smithi076 ceph-mon[35642]: osdmap e17: 3 total, 2 up, 3 in 2024-01-25T16:01:13.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:13.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:13 smithi076 ceph-mon[35642]: from='osd.2 ' entity='osd.2' 2024-01-25T16:01:14.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi152 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 17080M 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] boot 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: osdmap e18: 3 total, 3 up, 3 in 2024-01-25T16:01:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi152 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 17080M 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] boot 2024-01-25T16:01:14.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: osdmap e18: 3 total, 3 up, 3 in 2024-01-25T16:01:14.885 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:01:14.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:01:14.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi152 2024-01-25T16:01:14.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 17080M 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.920 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: osd.2 [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] boot 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: osdmap e18: 3 total, 3 up, 3 in 2024-01-25T16:01:14.921 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:01:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 -- 172.21.15.33:0/4250706592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 msgr2=0x7f0838108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 --2- 172.21.15.33:0/4250706592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838108c60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f083000b0a0 tx=0x7f083000b3b0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 -- 172.21.15.33:0/4250706592 shutdown_connections 2024-01-25T16:01:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 --2- 172.21.15.33:0/4250706592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838108c60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 --2- 172.21.15.33:0/4250706592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838101cc0 0x7f08381020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 --2- 172.21.15.33:0/4250706592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838100a00 0x7f0838100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 -- 172.21.15.33:0/4250706592 >> 172.21.15.33:0/4250706592 conn(0x7f08380fb470 msgr2=0x7f08380fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 -- 172.21.15.33:0/4250706592 shutdown_connections 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.937+0000 7f083df3f700 1 -- 172.21.15.33:0/4250706592 wait complete. 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 Processor -- start 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 -- start start 2024-01-25T16:01:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 0x7f0838197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 0x7f083819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819f2d0 con 0x7f08381013a0 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819f410 con 0x7f0838101cc0 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f083df3f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083819f5a0 con 0x7f0838100a00 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f0836ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f0836ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54142/0 (socket says 172.21.15.33:54142) 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.938+0000 7f0836ffd700 1 -- 172.21.15.33:0/3635803862 learned_addr learned my addr 172.21.15.33:0/3635803862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f08377fe700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 0x7f0838197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0837fff700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 0x7f083819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:14.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 -- 172.21.15.33:0/3635803862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 msgr2=0x7f0838197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 0x7f0838197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 -- 172.21.15.33:0/3635803862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 msgr2=0x7f083819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 0x7f083819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 -- 172.21.15.33:0/3635803862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830009d00 con 0x7f08381013a0 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f08377fe700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 0x7f0838197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0837fff700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 0x7f083819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0836ffd700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c9e0 tx=0x7f082c00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c007ac0 con 0x7f08381013a0 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f082c007c20 con 0x7f08381013a0 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083819f880 con 0x7f08381013a0 2024-01-25T16:01:14.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c018660 con 0x7f08381013a0 2024-01-25T16:01:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.939+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083819fd80 con 0x7f08381013a0 2024-01-25T16:01:14.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.940+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0838107b00 con 0x7f08381013a0 2024-01-25T16:01:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.941+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) v1 ==== 135839+0+0 (secure 0 0 0) 0x7f082c00f450 con 0x7f08381013a0 2024-01-25T16:01:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.942+0000 7f0834ff9700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 0x7f08200a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.942+0000 7f08377fe700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 0x7f08200a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.942+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2851+0+0 (secure 0 0 0) 0x7f082c0cb490 con 0x7f08381013a0 2024-01-25T16:01:14.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.942+0000 7f08377fe700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 0x7f08200a37a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0828005950 tx=0x7f08280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:01:14.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:14.946+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f082c095f40 con 0x7f08381013a0 2024-01-25T16:01:15.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:15.158+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f083802d4a0 con 0x7f08200a12e0 2024-01-25T16:01:16.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:15 smithi033 ceph-mon[31417]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:16.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:16.044 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:16.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:16.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:15 smithi033 ceph-mon[31417]: osdmap e19: 3 total, 3 up, 3 in 2024-01-25T16:01:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:15 smithi152 ceph-mon[36015]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:16.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:15 smithi152 ceph-mon[36015]: osdmap e19: 3 total, 3 up, 3 in 2024-01-25T16:01:16.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:15 smithi076 ceph-mon[35642]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:16.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:16.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:16.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:16.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:15 smithi076 ceph-mon[35642]: osdmap e19: 3 total, 3 up, 3 in 2024-01-25T16:01:17.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:16 smithi033 ceph-mon[31417]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:17.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:16 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:16 smithi152 ceph-mon[36015]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:16 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:16 smithi076 ceph-mon[35642]: pgmap v83: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:16 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:18.314 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50765]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:18.314 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:18.315 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50765]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:18.315 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50765]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: osdmap e20: 3 total, 3 up, 3 in 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]: dispatch 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]': finished 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: osdmap e21: 4 total, 3 up, 4 in 2024-01-25T16:01:18.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: osdmap e20: 3 total, 3 up, 3 in 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]: dispatch 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]': finished 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: osdmap e21: 4 total, 3 up, 4 in 2024-01-25T16:01:18.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: osdmap e20: 3 total, 3 up, 3 in 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]: dispatch 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4214712014' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d47d7676-603d-43dd-84d5-278ad2c00170"}]': finished 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: osdmap e21: 4 total, 3 up, 4 in 2024-01-25T16:01:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:18.814 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50787]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:18.814 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50787]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:18.814 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50787]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:18.815 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:01:18 smithi033 sudo[50787]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:18.888 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44283]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:18.888 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:18.888 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44283]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:18.888 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44283]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:19.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:19.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2370533503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:19.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: osdmap e22: 4 total, 3 up, 4 in 2024-01-25T16:01:19.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:19.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44286]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:19.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:19.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44286]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:19.169 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:01:18 smithi076 sudo[44286]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:19.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:19.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2370533503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: osdmap e22: 4 total, 3 up, 4 in 2024-01-25T16:01:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:19.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44989]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:19.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:19.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44989]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:19.383 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44989]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2370533503' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: osdmap e22: 4 total, 3 up, 4 in 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:19.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:19.883 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44992]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:19.883 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44992]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:19.883 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44992]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:19.883 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:01:19 smithi152 sudo[44992]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:19.968 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51210]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:19.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51210]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:19.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51210]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:19.969 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51210]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:19.979+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f082c096310 con 0x7f08381013a0 2024-01-25T16:01:20.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51214]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:20.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51214]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:20.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51214]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:20.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:19 smithi033 sudo[51214]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:20.467 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44995]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:20.467 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44995]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:20.467 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44995]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:20.467 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44995]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:20.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:20.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:20.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:20 smithi033 ceph-mon[31417]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44998]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44998]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44998]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 sudo[44998]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:20 smithi152 ceph-mon[36015]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:20.961 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44289]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44289]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44289]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44292]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44292]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44292]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-01-25T16:01:20.962 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 sudo[44292]: pam_unix(sudo:session): session closed for user root 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:21.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:20 smithi076 ceph-mon[35642]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:21.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:21.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:21 smithi033 ceph-mon[31417]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:21.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:21 smithi152 ceph-mon[36015]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:21.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-01-25T16:01:21.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:01:21.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:01:21.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:01:21.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:21 smithi076 ceph-mon[35642]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-01-25T16:01:22.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:22 smithi033 ceph-mon[31417]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:22.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:22 smithi152 ceph-mon[36015]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:22.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:22 smithi076 ceph-mon[35642]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:25.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:25 smithi152 ceph-mon[36015]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:25 smithi076 ceph-mon[35642]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:25 smithi033 ceph-mon[31417]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:26.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-25T16:01:26.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:26.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:26 smithi033 ceph-mon[31417]: Deploying daemon osd.3 on smithi033 2024-01-25T16:01:26.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-25T16:01:26.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:26.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:26 smithi152 ceph-mon[36015]: Deploying daemon osd.3 on smithi033 2024-01-25T16:01:26.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-01-25T16:01:26.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:26.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:26 smithi076 ceph-mon[35642]: Deploying daemon osd.3 on smithi033 2024-01-25T16:01:27.280 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:27 smithi033 ceph-mon[31417]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:27.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:27 smithi152 ceph-mon[36015]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:27 smithi076 ceph-mon[35642]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:29.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:29 smithi152 ceph-mon[36015]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:29 smithi076 ceph-mon[35642]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:29 smithi033 ceph-mon[31417]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:31.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:30 smithi033 ceph-mon[31417]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:31.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:30 smithi152 ceph-mon[36015]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:30 smithi076 ceph-mon[35642]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:33 smithi033 ceph-mon[31417]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:33.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:33 smithi152 ceph-mon[36015]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:33.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:33 smithi076 ceph-mon[35642]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:34.115 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-01-25T16:01:34.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.111+0000 7f0834ff9700 1 -- 172.21.15.33:0/3635803862 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f083802d4a0 con 0x7f08200a12e0 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 msgr2=0x7f08200a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 0x7f08200a37a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0828005950 tx=0x7f08280058e0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 msgr2=0x7f0838197c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c9e0 tx=0x7f082c00cda0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 shutdown_connections 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08200a12e0 0x7f08200a37a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0838101cc0 0x7f083819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0838100a00 0x7f0838197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 --2- 172.21.15.33:0/3635803862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08381013a0 0x7f0838197c80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 >> 172.21.15.33:0/3635803862 conn(0x7f08380fb470 msgr2=0x7f0838104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:01:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 shutdown_connections 2024-01-25T16:01:34.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:01:34.113+0000 7f083df3f700 1 -- 172.21.15.33:0/3635803862 wait complete. 2024-01-25T16:01:34.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.468 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:34.469 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:34.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:34.798 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.3.service 2024-01-25T16:01:34.802 INFO:tasks.cephadm:Deploying osd.4 on smithi076 with /dev/vg_nvme/lv_3... 2024-01-25T16:01:34.802 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-25T16:01:35.054 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:01:35.473 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:35 smithi076 ceph-mon[35642]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-25T16:01:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:35 smithi033 ceph-mon[31417]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-25T16:01:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:35 smithi152 ceph-mon[36015]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-01-25T16:01:36.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:36 smithi033 ceph-mon[31417]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-25T16:01:36.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:36 smithi033 ceph-mon[31417]: osdmap e23: 4 total, 3 up, 4 in 2024-01-25T16:01:36.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:36 smithi033 ceph-mon[31417]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:01:36.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:36.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:36 smithi033 ceph-mon[31417]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:36.628 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:36 smithi076 ceph-mon[35642]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-25T16:01:36.628 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:36 smithi076 ceph-mon[35642]: osdmap e23: 4 total, 3 up, 4 in 2024-01-25T16:01:36.628 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:36 smithi076 ceph-mon[35642]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:01:36.628 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:36.628 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:36 smithi076 ceph-mon[35642]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:36 smithi152 ceph-mon[36015]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-01-25T16:01:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:36 smithi152 ceph-mon[36015]: osdmap e23: 4 total, 3 up, 4 in 2024-01-25T16:01:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:36 smithi152 ceph-mon[36015]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:01:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:36 smithi152 ceph-mon[36015]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 95 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113175 s, 927 MB/s 2024-01-25T16:01:36.933 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-01-25T16:01:37.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-mon[31417]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:01:37.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-mon[31417]: osdmap e24: 4 total, 3 up, 4 in 2024-01-25T16:01:37.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.465 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-mon[31417]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' 2024-01-25T16:01:37.466 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:01:37.117+0000 7f155d6ff700 -1 osd.3 0 waiting for initial osdmap 2024-01-25T16:01:37.466 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:01:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:01:37.125+0000 7f15544ef700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:01:37.532 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:37 smithi076 ceph-mon[35642]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:01:37.532 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:37 smithi076 ceph-mon[35642]: osdmap e24: 4 total, 3 up, 4 in 2024-01-25T16:01:37.532 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.532 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.532 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:37 smithi076 ceph-mon[35642]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' 2024-01-25T16:01:37.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_3 2024-01-25T16:01:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:37 smithi152 ceph-mon[36015]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:01:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:37 smithi152 ceph-mon[36015]: osdmap e24: 4 total, 3 up, 4 in 2024-01-25T16:01:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:37 smithi152 ceph-mon[36015]: from='osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703]' entity='osd.3' 2024-01-25T16:01:37.799 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:01:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:01:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] boot 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: osdmap e25: 4 total, 4 up, 4 in 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi033 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 8546M 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:38.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:38 smithi076 ceph-mon[35642]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:38.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] boot 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: osdmap e25: 4 total, 4 up, 4 in 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi033 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 8546M 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:38.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:38.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:38 smithi033 ceph-mon[31417]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: osd.3 [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] boot 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: osdmap e25: 4 total, 4 up, 4 in 2024-01-25T16:01:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi033 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 8546M 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:38 smithi152 ceph-mon[36015]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 -- 172.21.15.76:0/3237128534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794105440 msgr2=0x7f27941058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 --2- 172.21.15.76:0/3237128534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794105440 0x7f27941058c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f278c00b0a0 tx=0x7f278c00b3b0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 -- 172.21.15.76:0/3237128534 shutdown_connections 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 --2- 172.21.15.76:0/3237128534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794105440 0x7f27941058c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 --2- 172.21.15.76:0/3237128534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794100100 0x7f2794102530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 --2- 172.21.15.76:0/3237128534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794102a70 0x7f2794104f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 -- 172.21.15.76:0/3237128534 >> 172.21.15.76:0/3237128534 conn(0x7f27940fbd40 msgr2=0x7f27940fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 -- 172.21.15.76:0/3237128534 shutdown_connections 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.257+0000 7f279a44c700 1 -- 172.21.15.76:0/3237128534 wait complete. 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 Processor -- start 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 -- start start 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 0x7f279419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 0x7f27941a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941a59a0 con 0x7f2794100100 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941a5ae0 con 0x7f2794102a70 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f279a44c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27941a5c70 con 0x7f2794105440 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:39.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35474/0 (socket says 172.21.15.76:35474) 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 -- 172.21.15.76:0/1402284514 learned_addr learned my addr 172.21.15.76:0/1402284514 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f2793fff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 0x7f279419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 -- 172.21.15.76:0/1402284514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 msgr2=0x7f27941a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f27989e9700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 0x7f27941a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 0x7f27941a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 -- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 msgr2=0x7f279419de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 0x7f279419de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 -- 172.21.15.76:0/1402284514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c009d00 con 0x7f2794102a70 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f2793fff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 0x7f279419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f27989e9700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 0x7f27941a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.258+0000 7f278bfff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d950 tx=0x7f277c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.259+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c009b40 con 0x7f2794102a70 2024-01-25T16:01:39.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.259+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f277c009ca0 con 0x7f2794102a70 2024-01-25T16:01:39.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.259+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27941a5f50 con 0x7f2794102a70 2024-01-25T16:01:39.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.259+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00f620 con 0x7f2794102a70 2024-01-25T16:01:39.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.259+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27941a6440 con 0x7f2794102a70 2024-01-25T16:01:39.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.260+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27940684a0 con 0x7f2794102a70 2024-01-25T16:01:39.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.261+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f277c010460 con 0x7f2794102a70 2024-01-25T16:01:39.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.261+0000 7f2791ffb700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 0x7f27800a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:01:39.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.262+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f277c0cb8c0 con 0x7f2794102a70 2024-01-25T16:01:39.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.262+0000 7f2793fff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 0x7f27800a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:01:39.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.262+0000 7f2793fff700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 0x7f27800a3830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2784007900 tx=0x7f2784008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:01:39.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.265+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f277c095f40 con 0x7f2794102a70 2024-01-25T16:01:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:39 smithi076 ceph-mon[35642]: osdmap e26: 4 total, 4 up, 4 in 2024-01-25T16:01:39.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:39.466+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f279402d630 con 0x7f27800a1370 2024-01-25T16:01:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:39 smithi033 ceph-mon[31417]: osdmap e26: 4 total, 4 up, 4 in 2024-01-25T16:01:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:39 smithi152 ceph-mon[36015]: osdmap e26: 4 total, 4 up, 4 in 2024-01-25T16:01:40.360 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: osdmap e27: 4 total, 4 up, 4 in 2024-01-25T16:01:40.360 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:40.360 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:40.360 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:40.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:40.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: osdmap e28: 4 total, 4 up, 4 in 2024-01-25T16:01:40.361 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:40 smithi076 ceph-mon[35642]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: osdmap e27: 4 total, 4 up, 4 in 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:40.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: osdmap e28: 4 total, 4 up, 4 in 2024-01-25T16:01:40.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:40 smithi033 ceph-mon[31417]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: osdmap e27: 4 total, 4 up, 4 in 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: osdmap e28: 4 total, 4 up, 4 in 2024-01-25T16:01:40.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:40 smithi152 ceph-mon[36015]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:42.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:41 smithi033 ceph-mon[31417]: osdmap e29: 4 total, 4 up, 4 in 2024-01-25T16:01:42.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:41 smithi152 ceph-mon[36015]: osdmap e29: 4 total, 4 up, 4 in 2024-01-25T16:01:42.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:41 smithi076 ceph-mon[35642]: osdmap e29: 4 total, 4 up, 4 in 2024-01-25T16:01:43.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-25T16:01:43.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/2665104379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]': finished 2024-01-25T16:01:43.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: osdmap e30: 5 total, 4 up, 5 in 2024-01-25T16:01:43.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:01:43.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/3341843800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/2665104379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]': finished 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: osdmap e30: 5 total, 4 up, 5 in 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:01:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/3341843800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/2665104379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]: dispatch 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd"}]': finished 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: osdmap e30: 5 total, 4 up, 5 in 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:01:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/3341843800' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:01:45.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:45 smithi152 ceph-mon[36015]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-01-25T16:01:45.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:45 smithi076 ceph-mon[35642]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-01-25T16:01:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:45 smithi033 ceph-mon[31417]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-01-25T16:01:47.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:47 smithi076 ceph-mon[35642]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-01-25T16:01:47.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:47 smithi152 ceph-mon[36015]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-01-25T16:01:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:47 smithi033 ceph-mon[31417]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-01-25T16:01:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:49 smithi152 ceph-mon[36015]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:01:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:49 smithi076 ceph-mon[35642]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:01:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:49 smithi033 ceph-mon[31417]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:01:50.369 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-25T16:01:50.369 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:50.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-25T16:01:50.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:50.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-01-25T16:01:50.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:51.383 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:51 smithi076 ceph-mon[35642]: Deploying daemon osd.4 on smithi076 2024-01-25T16:01:51.383 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:51 smithi076 ceph-mon[35642]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-01-25T16:01:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:51 smithi152 ceph-mon[36015]: Deploying daemon osd.4 on smithi076 2024-01-25T16:01:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:51 smithi152 ceph-mon[36015]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-01-25T16:01:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:51 smithi033 ceph-mon[31417]: Deploying daemon osd.4 on smithi076 2024-01-25T16:01:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:51 smithi033 ceph-mon[31417]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-01-25T16:01:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:53 smithi152 ceph-mon[36015]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:53 smithi076 ceph-mon[35642]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:53 smithi033 ceph-mon[31417]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:55 smithi152 ceph-mon[36015]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:55 smithi076 ceph-mon[35642]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:55 smithi033 ceph-mon[31417]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:57.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:57 smithi152 ceph-mon[36015]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:57 smithi076 ceph-mon[35642]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:57 smithi033 ceph-mon[31417]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.530+0000 7f2791ffb700 1 -- 172.21.15.76:0/1402284514 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f279402d630 con 0x7f27800a1370 2024-01-25T16:01:58.534 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 4 on host 'smithi076' 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 msgr2=0x7f27800a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 0x7f27800a3830 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2784007900 tx=0x7f2784008040 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 msgr2=0x7f279419e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f277c00d950 tx=0x7f277c00dd10 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 shutdown_connections 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.532+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f27800a1370 0x7f27800a3830 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2794105440 0x7f27941a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2794100100 0x7f279419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 --2- 172.21.15.76:0/1402284514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2794102a70 0x7f279419e350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 >> 172.21.15.76:0/1402284514 conn(0x7f27940fbd40 msgr2=0x7f2794100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:01:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 shutdown_connections 2024-01-25T16:01:58.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:01:58.533+0000 7f279a44c700 1 -- 172.21.15.76:0/1402284514 wait complete. 2024-01-25T16:01:58.821 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:58.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:58 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:59.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:58 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:59.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:01:59.178 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:58 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:01:59.180 DEBUG:teuthology.orchestra.run.smithi076:osd.4> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.4.service 2024-01-25T16:01:59.186 INFO:tasks.cephadm:Deploying osd.5 on smithi152 with /dev/vg_nvme/lv_3... 2024-01-25T16:01:59.186 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-01-25T16:01:59.431 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:01:59.872 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:59 smithi152 ceph-mon[36015]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:01:59.872 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:01:59 smithi152 ceph-mon[36015]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:02:00.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:59 smithi033 ceph-mon[31417]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:02:00.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:01:59 smithi033 ceph-mon[31417]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:02:00.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:59 smithi076 ceph-mon[35642]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:02:00.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:01:59 smithi076 ceph-mon[35642]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-01-25T16:02:00.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-25T16:02:00.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: osdmap e31: 5 total, 4 up, 5 in 2024-01-25T16:02:00.822 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:00.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:00.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:00.823 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-mon[35642]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:02:00.823 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:02:00.724+0000 7f7285fe7700 -1 osd.4 0 waiting for initial osdmap 2024-01-25T16:02:00.823 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:02:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:02:00.732+0000 7f72825e2700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:02:01.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-25T16:02:01.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: osdmap e31: 5 total, 4 up, 5 in 2024-01-25T16:02:01.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:01.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:01.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:01.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:00 smithi033 ceph-mon[31417]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: osdmap e31: 5 total, 4 up, 5 in 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: from='osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:02:01.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:00 smithi152 ceph-mon[36015]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 124 MiB used, 357 GiB / 358 GiB avail 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115069 s, 911 MB/s 2024-01-25T16:02:01.269 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-01-25T16:02:01.855 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:01 smithi152 ceph-mon[36015]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:02:01.856 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:01 smithi152 ceph-mon[36015]: osdmap e32: 5 total, 4 up, 5 in 2024-01-25T16:02:01.856 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:01.856 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:01.885 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_3 2024-01-25T16:02:02.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:01 smithi033 ceph-mon[31417]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:02:02.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:01 smithi033 ceph-mon[31417]: osdmap e32: 5 total, 4 up, 5 in 2024-01-25T16:02:02.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:02.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:02.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:01 smithi076 ceph-mon[35642]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:02:02.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:01 smithi076 ceph-mon[35642]: osdmap e32: 5 total, 4 up, 5 in 2024-01-25T16:02:02.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:02.119 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:02.121 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:02:02.795 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:02:02.795 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:02:02.795 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] boot 2024-01-25T16:02:02.795 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: osdmap e33: 5 total, 5 up, 5 in 2024-01-25T16:02:02.795 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi076 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 8540M 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:02.796 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] boot 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: osdmap e33: 5 total, 5 up, 5 in 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi076 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 8540M 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:03.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: osd.4 [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] boot 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: osdmap e33: 5 total, 5 up, 5 in 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi076 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:03.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:03.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 8540M 2024-01-25T16:02:03.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:03.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:03.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:03.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 -- 172.21.15.152:0/1571968208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96141019b0 msgr2=0x7f96141082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:03.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 --2- 172.21.15.152:0/1571968208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96141019b0 0x7f96141082d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f960c00c210 tx=0x7f960c00c520 comp rx=0 tx=0).stop 2024-01-25T16:02:03.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 -- 172.21.15.152:0/1571968208 shutdown_connections 2024-01-25T16:02:03.561 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 --2- 172.21.15.152:0/1571968208 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96141019b0 0x7f96141082d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 --2- 172.21.15.152:0/1571968208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96140ffe90 0x7f96141002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 --2- 172.21.15.152:0/1571968208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9614100ff0 0x7f9614101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 -- 172.21.15.152:0/1571968208 >> 172.21.15.152:0/1571968208 conn(0x7f96140fb450 msgr2=0x7f96140fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 -- 172.21.15.152:0/1571968208 shutdown_connections 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.557+0000 7f961bb04700 1 -- 172.21.15.152:0/1571968208 wait complete. 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 Processor -- start 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 -- start start 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9614100ff0 0x7f9614195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96141019b0 0x7f961419c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419cea0 con 0x7f96141019b0 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419cfe0 con 0x7f9614100ff0 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961bb04700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419d170 con 0x7f96140ffe90 2024-01-25T16:02:03.562 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:03.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:59302/0 (socket says 172.21.15.152:59302) 2024-01-25T16:02:03.563 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 -- 172.21.15.152:0/3715438324 learned_addr learned my addr 172.21.15.152:0/3715438324 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T16:02:03.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961909f700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9614100ff0 0x7f9614195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:03.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f961a0a1700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96141019b0 0x7f961419c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:03.567 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 -- 172.21.15.152:0/3715438324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9614100ff0 msgr2=0x7f9614195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9614100ff0 0x7f9614195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 -- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96141019b0 msgr2=0x7f961419c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96141019b0 0x7f961419c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.558+0000 7f96198a0700 1 -- 172.21.15.152:0/3715438324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960c00ae10 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f96198a0700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9d0 tx=0x7f960400ed90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604009790 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961419d450 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f960400cbd0 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604018680 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.559+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961419d9a0 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.560+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96141071e0 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.561+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f9604010430 con 0x7f96140ffe90 2024-01-25T16:02:03.568 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.561+0000 7f960affd700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 0x7f96000a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:03.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.562+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f9604014070 con 0x7f96140ffe90 2024-01-25T16:02:03.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.562+0000 7f961909f700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 0x7f96000a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:03.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.562+0000 7f961909f700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 0x7f96000a3830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9610005fd0 tx=0x7f9610005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:03.569 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.565+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9604096500 con 0x7f96140ffe90 2024-01-25T16:02:03.770 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:03.767+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f961402d4d0 con 0x7f96000a1370 2024-01-25T16:02:04.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:03 smithi033 ceph-mon[31417]: osdmap e34: 5 total, 5 up, 5 in 2024-01-25T16:02:04.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:03 smithi152 ceph-mon[36015]: osdmap e34: 5 total, 5 up, 5 in 2024-01-25T16:02:04.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:03 smithi076 ceph-mon[35642]: osdmap e34: 5 total, 5 up, 5 in 2024-01-25T16:02:05.013 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:04 smithi152 ceph-mon[36015]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:05.013 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:05.013 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:05.013 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:05.013 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:04 smithi152 ceph-mon[36015]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:05.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:04 smithi033 ceph-mon[31417]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:05.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:05.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:05.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:05.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:04 smithi033 ceph-mon[31417]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:05.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:04 smithi076 ceph-mon[35642]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:05.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:05.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:05.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:05.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:04 smithi076 ceph-mon[35642]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/1055576249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]': finished 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: osdmap e35: 6 total, 5 up, 6 in 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:07.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:07 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/361403702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/1055576249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]': finished 2024-01-25T16:02:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: osdmap e35: 6 total, 5 up, 6 in 2024-01-25T16:02:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:07 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/361403702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/1055576249' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]: dispatch 2024-01-25T16:02:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "722b8a67-2c0c-46f4-bcac-37587042f804"}]': finished 2024-01-25T16:02:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: osdmap e35: 6 total, 5 up, 6 in 2024-01-25T16:02:07.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:07.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:07 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/361403702' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:09.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:09 smithi152 ceph-mon[36015]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:09.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:09 smithi076 ceph-mon[35642]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:09 smithi033 ceph-mon[31417]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:11.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:10 smithi152 ceph-mon[36015]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:11.264 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:10 smithi076 ceph-mon[35642]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:10 smithi033 ceph-mon[31417]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:13.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:13 smithi152 ceph-mon[36015]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:13 smithi076 ceph-mon[35642]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:13 smithi033 ceph-mon[31417]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:14.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-25T16:02:14.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-25T16:02:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:14.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-01-25T16:02:14.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:15 smithi152 ceph-mon[36015]: Deploying daemon osd.5 on smithi152 2024-01-25T16:02:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:15 smithi152 ceph-mon[36015]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:15 smithi076 ceph-mon[35642]: Deploying daemon osd.5 on smithi152 2024-01-25T16:02:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:15 smithi076 ceph-mon[35642]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:15 smithi033 ceph-mon[31417]: Deploying daemon osd.5 on smithi152 2024-01-25T16:02:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:15 smithi033 ceph-mon[31417]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:17 smithi152 ceph-mon[36015]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:17 smithi076 ceph-mon[35642]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:17 smithi033 ceph-mon[31417]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:19 smithi152 ceph-mon[36015]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:19 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:19 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:19 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:19 smithi076 ceph-mon[35642]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:19 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:19 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:19 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:19 smithi033 ceph-mon[31417]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:19.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:19 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:19 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:19.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:19 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:21.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:20 smithi033 ceph-mon[31417]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:21.378 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:20 smithi152 ceph-mon[36015]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:20 smithi076 ceph-mon[35642]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:22.337 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 5 on host 'smithi152' 2024-01-25T16:02:22.337 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.333+0000 7f960affd700 1 -- 172.21.15.152:0/3715438324 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f961402d4d0 con 0x7f96000a1370 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 msgr2=0x7f96000a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 0x7f96000a3830 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9610005fd0 tx=0x7f9610005f60 comp rx=0 tx=0).stop 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 msgr2=0x7f9614195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9d0 tx=0x7f960400ed90 comp rx=0 tx=0).stop 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 shutdown_connections 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f96000a1370 0x7f96000a3830 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96141019b0 0x7f961419c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:22.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f96140ffe90 0x7f9614195310 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:22.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 --2- 172.21.15.152:0/3715438324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9614100ff0 0x7f9614195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:22.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 >> 172.21.15.152:0/3715438324 conn(0x7f96140fb450 msgr2=0x7f9614104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:02:22.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 shutdown_connections 2024-01-25T16:02:22.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:02:22.336+0000 7f961bb04700 1 -- 172.21.15.152:0/3715438324 wait complete. 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:22 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:22.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:22 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:22.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:22.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:22 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:22.961 DEBUG:teuthology.orchestra.run.smithi152:osd.5> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.5.service 2024-01-25T16:02:22.964 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-01-25T16:02:22.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-25T16:02:23.212 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:02:23.599 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:23 smithi033 ceph-mon[31417]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:23.600 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:23 smithi033 ceph-mon[31417]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:23.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:23 smithi152 ceph-mon[36015]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:23.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:23 smithi152 ceph-mon[36015]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:23.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:23 smithi076 ceph-mon[35642]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:23.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:23 smithi076 ceph-mon[35642]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: osdmap e36: 6 total, 5 up, 6 in 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:24 smithi033 ceph-mon[31417]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: osdmap e36: 6 total, 5 up, 6 in 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:24 smithi152 ceph-mon[36015]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:24.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-01-25T16:02:24.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: osdmap e36: 6 total, 5 up, 6 in 2024-01-25T16:02:24.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:24.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: from='osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:02:24.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:24 smithi076 ceph-mon[35642]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110965 s, 945 MB/s 2024-01-25T16:02:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-01-25T16:02:25.741 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: osdmap e37: 6 total, 5 up, 6 in 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: osdmap e38: 6 total, 5 up, 6 in 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='osd.5 ' entity='osd.5' 2024-01-25T16:02:25.742 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.742 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:02:25.415+0000 7fecde4ae700 -1 osd.5 0 waiting for initial osdmap 2024-01-25T16:02:25.742 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:02:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:02:25.425+0000 7fecd6aa1700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:02:25.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:02:25.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: osdmap e37: 6 total, 5 up, 6 in 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: osdmap e38: 6 total, 5 up, 6 in 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='osd.5 ' entity='osd.5' 2024-01-25T16:02:25.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: osdmap e37: 6 total, 5 up, 6 in 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: osdmap e38: 6 total, 5 up, 6 in 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='osd.5 ' entity='osd.5' 2024-01-25T16:02:25.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:25.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-01-25T16:02:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi152 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 8540M 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.041 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] boot 2024-01-25T16:02:27.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: osdmap e39: 6 total, 6 up, 6 in 2024-01-25T16:02:27.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:27.042 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:26 smithi033 ceph-mon[31417]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi152 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 8540M 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] boot 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: osdmap e39: 6 total, 6 up, 6 in 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:27.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:26 smithi152 ceph-mon[36015]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:27.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:02:27.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi152 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 8540M 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: osd.5 [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] boot 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: osdmap e39: 6 total, 6 up, 6 in 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:02:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:26 smithi076 ceph-mon[35642]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-01-25T16:02:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 -- 172.21.15.33:0/2791729525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c100710 msgr2=0x7f903c100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 --2- 172.21.15.33:0/2791729525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c100710 0x7f903c100b30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9034009990 tx=0x7f9034009ca0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 -- 172.21.15.33:0/2791729525 shutdown_connections 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 --2- 172.21.15.33:0/2791729525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 0x7f903c108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 --2- 172.21.15.33:0/2791729525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c100710 0x7f903c100b30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 --2- 172.21.15.33:0/2791729525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c101910 0x7f903c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 -- 172.21.15.33:0/2791729525 >> 172.21.15.33:0/2791729525 conn(0x7f903c0fbcd0 msgr2=0x7f903c0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 -- 172.21.15.33:0/2791729525 shutdown_connections 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.796+0000 7f90441f6700 1 -- 172.21.15.33:0/2791729525 wait complete. 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 Processor -- start 2024-01-25T16:02:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 -- start start 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c100710 0x7f903c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 0x7f903c19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19f1f0 con 0x7f903c101910 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19f330 con 0x7f903c1022b0 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f90441f6700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c19f4c0 con 0x7f903c100710 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33794/0 (socket says 172.21.15.33:33794) 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 -- 172.21.15.33:0/3532028316 learned_addr learned my addr 172.21.15.33:0/3532028316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041f92700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c100710 0x7f903c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9042793700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 0x7f903c19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 -- 172.21.15.33:0/3532028316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c100710 msgr2=0x7f903c197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c100710 0x7f903c197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 -- 172.21.15.33:0/3532028316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 msgr2=0x7f903c19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 0x7f903c19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.797+0000 7f9041791700 1 -- 172.21.15.33:0/3532028316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90340095f0 con 0x7f903c101910 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f9041791700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f902c00d980 tx=0x7f902c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0098d0 con 0x7f903c101910 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f902c010460 con 0x7f903c101910 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c19f7a0 con 0x7f903c101910 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c00f620 con 0x7f903c101910 2024-01-25T16:02:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.798+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c19fcf0 con 0x7f903c101910 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.799+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c107ad0 con 0x7f903c101910 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.800+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f902c010860 con 0x7f903c101910 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.800+0000 7f9032ffd700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 0x7f90280a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.801+0000 7f9041f92700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 0x7f90280a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.801+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f902c0cb9a0 con 0x7f903c101910 2024-01-25T16:02:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.801+0000 7f9041f92700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 0x7f90280a38b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f903400b5c0 tx=0x7f9034003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:27.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:27.805+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f902c095cc0 con 0x7f903c101910 2024-01-25T16:02:28.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:28.019+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f903c02d540 con 0x7f90280a13f0 2024-01-25T16:02:28.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:28 smithi033 ceph-mon[31417]: osdmap e40: 6 total, 6 up, 6 in 2024-01-25T16:02:28.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:28 smithi152 ceph-mon[36015]: osdmap e40: 6 total, 6 up, 6 in 2024-01-25T16:02:28.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:28 smithi076 ceph-mon[35642]: osdmap e40: 6 total, 6 up, 6 in 2024-01-25T16:02:29.284 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: osdmap e41: 6 total, 6 up, 6 in 2024-01-25T16:02:29.284 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:29.284 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:29.284 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:29.285 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:29.285 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:29 smithi152 ceph-mon[36015]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:29.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: osdmap e41: 6 total, 6 up, 6 in 2024-01-25T16:02:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:29.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:29 smithi033 ceph-mon[31417]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: osdmap e41: 6 total, 6 up, 6 in 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:29.320 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:29 smithi076 ceph-mon[35642]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:30.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:30 smithi033 ceph-mon[31417]: osdmap e42: 6 total, 6 up, 6 in 2024-01-25T16:02:30.333 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:30 smithi076 ceph-mon[35642]: osdmap e42: 6 total, 6 up, 6 in 2024-01-25T16:02:30.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:30 smithi152 ceph-mon[36015]: osdmap e42: 6 total, 6 up, 6 in 2024-01-25T16:02:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: osdmap e43: 6 total, 6 up, 6 in 2024-01-25T16:02:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]: dispatch 2024-01-25T16:02:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]': finished 2024-01-25T16:02:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: osdmap e44: 7 total, 6 up, 7 in 2024-01-25T16:02:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: osdmap e43: 6 total, 6 up, 6 in 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]: dispatch 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]': finished 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: osdmap e44: 7 total, 6 up, 7 in 2024-01-25T16:02:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: osdmap e43: 6 total, 6 up, 6 in 2024-01-25T16:02:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]: dispatch 2024-01-25T16:02:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2902060113' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5c38c5ba-1402-4433-9924-49b144ff520d"}]': finished 2024-01-25T16:02:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: osdmap e44: 7 total, 6 up, 7 in 2024-01-25T16:02:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:32.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:32 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1764554778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:32.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:32 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1764554778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:32.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:32 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1764554778' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:33 smithi033 ceph-mon[31417]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:02:33.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:33 smithi152 ceph-mon[36015]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:02:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:33 smithi076 ceph-mon[35642]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:02:35.266 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:35 smithi033 ceph-mon[31417]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:02:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:35 smithi152 ceph-mon[36015]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:02:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:35 smithi076 ceph-mon[35642]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:02:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:37 smithi152 ceph-mon[36015]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:02:37.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:37 smithi076 ceph-mon[35642]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:02:37.523 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:37 smithi033 ceph-mon[31417]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:02:38.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-25T16:02:38.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-25T16:02:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-01-25T16:02:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:39.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:39 smithi033 ceph-mon[31417]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:02:39.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:39 smithi033 ceph-mon[31417]: Deploying daemon osd.6 on smithi033 2024-01-25T16:02:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:39 smithi152 ceph-mon[36015]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:02:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:39 smithi152 ceph-mon[36015]: Deploying daemon osd.6 on smithi033 2024-01-25T16:02:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:39 smithi076 ceph-mon[35642]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:02:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:39 smithi076 ceph-mon[35642]: Deploying daemon osd.6 on smithi033 2024-01-25T16:02:41.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:41 smithi033 ceph-mon[31417]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:02:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:41 smithi152 ceph-mon[36015]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:02:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:41 smithi076 ceph-mon[35642]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:02:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:43 smithi152 ceph-mon[36015]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:02:43.398 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:43 smithi033 ceph-mon[31417]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:02:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:43 smithi076 ceph-mon[35642]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:02:44.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:44.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:44 smithi152 ceph-mon[36015]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:44 smithi076 ceph-mon[35642]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:44.984 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:44 smithi033 ceph-mon[31417]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:47.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:47 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.384 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:47.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:47.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:47.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:47 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:47 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:47.478 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-01-25T16:02:47.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.475+0000 7f9032ffd700 1 -- 172.21.15.33:0/3532028316 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f903c02d540 con 0x7f90280a13f0 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 msgr2=0x7f90280a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 0x7f90280a38b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f903400b5c0 tx=0x7f9034003790 comp rx=0 tx=0).stop 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 msgr2=0x7f903c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f902c00d980 tx=0x7f902c00dc90 comp rx=0 tx=0).stop 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 shutdown_connections 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f90280a13f0 0x7f90280a38b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f903c1022b0 0x7f903c19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:47.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f903c100710 0x7f903c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 --2- 172.21.15.33:0/3532028316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f903c101910 0x7f903c197ba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 >> 172.21.15.33:0/3532028316 conn(0x7f903c0fbcd0 msgr2=0x7f903c104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:02:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 shutdown_connections 2024-01-25T16:02:47.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:02:47.477+0000 7f90441f6700 1 -- 172.21.15.33:0/3532028316 wait complete. 2024-01-25T16:02:48.120 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.6.service 2024-01-25T16:02:48.122 INFO:tasks.cephadm:Deploying osd.7 on smithi076 with /dev/vg_nvme/lv_2... 2024-01-25T16:02:48.123 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-25T16:02:48.372 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:02:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:48 smithi076 ceph-mon[35642]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-25T16:02:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:48 smithi076 ceph-mon[35642]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:48.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:48.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:48 smithi033 ceph-mon[31417]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-25T16:02:48.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:48 smithi033 ceph-mon[31417]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:48.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:48.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:02:48.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:48 smithi152 ceph-mon[36015]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-01-25T16:02:48.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:48 smithi152 ceph-mon[36015]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:49.539 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:49 smithi076 ceph-mon[35642]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-25T16:02:49.539 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:49 smithi076 ceph-mon[35642]: osdmap e45: 7 total, 6 up, 7 in 2024-01-25T16:02:49.539 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:49 smithi076 ceph-mon[35642]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:02:49.539 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:49.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-mon[31417]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-25T16:02:49.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-mon[31417]: osdmap e45: 7 total, 6 up, 7 in 2024-01-25T16:02:49.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-mon[31417]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:02:49.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:49.553 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:02:49.431+0000 7f8eb6f6f700 -1 osd.6 0 waiting for initial osdmap 2024-01-25T16:02:49.553 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:02:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:02:49.441+0000 7f8eb356a700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:02:49.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:49 smithi152 ceph-mon[36015]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-01-25T16:02:49.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:49 smithi152 ceph-mon[36015]: osdmap e45: 7 total, 6 up, 7 in 2024-01-25T16:02:49.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:49 smithi152 ceph-mon[36015]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:02:49.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114209 s, 918 MB/s 2024-01-25T16:02:50.358 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-01-25T16:02:50.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:50 smithi033 ceph-mon[31417]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:02:50.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:50 smithi033 ceph-mon[31417]: osdmap e46: 7 total, 6 up, 7 in 2024-01-25T16:02:50.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:50 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.693 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:50 smithi033 ceph-mon[31417]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:50.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:50 smithi076 ceph-mon[35642]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:02:50.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:50 smithi076 ceph-mon[35642]: osdmap e46: 7 total, 6 up, 7 in 2024-01-25T16:02:50.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:50 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.710 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:50 smithi076 ceph-mon[35642]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:50.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:50 smithi152 ceph-mon[36015]: from='osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:02:50.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:50 smithi152 ceph-mon[36015]: osdmap e46: 7 total, 6 up, 7 in 2024-01-25T16:02:50.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:50 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:50.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:50 smithi152 ceph-mon[36015]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-01-25T16:02:51.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_2 2024-01-25T16:02:51.280 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:02:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:02:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:02:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] boot 2024-01-25T16:02:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: osdmap e47: 7 total, 7 up, 7 in 2024-01-25T16:02:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi033 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 5697M 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:51 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:02:51.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] boot 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: osdmap e47: 7 total, 7 up, 7 in 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi033 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 5697M 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:51 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: osd.6 [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] boot 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: osdmap e47: 7 total, 7 up, 7 in 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi033 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 5697M 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:02:51.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:51 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:02:52.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:52 smithi076 ceph-mon[35642]: osdmap e48: 7 total, 7 up, 7 in 2024-01-25T16:02:52.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:52 smithi076 ceph-mon[35642]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:52.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:52 smithi033 ceph-mon[31417]: osdmap e48: 7 total, 7 up, 7 in 2024-01-25T16:02:52.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:52 smithi033 ceph-mon[31417]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:52.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:52 smithi152 ceph-mon[36015]: osdmap e48: 7 total, 7 up, 7 in 2024-01-25T16:02:52.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:52 smithi152 ceph-mon[36015]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- 172.21.15.76:0/3459169850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa010101910 msgr2=0x7fa010101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 --2- 172.21.15.76:0/3459169850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa010101910 0x7fa010101d70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa0040099c0 tx=0x7fa004009cd0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- 172.21.15.76:0/3459169850 shutdown_connections 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 --2- 172.21.15.76:0/3459169850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0101022b0 0x7fa010108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 --2- 172.21.15.76:0/3459169850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa010100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 --2- 172.21.15.76:0/3459169850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa010101910 0x7fa010101d70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- 172.21.15.76:0/3459169850 >> 172.21.15.76:0/3459169850 conn(0x7fa0100fbcd0 msgr2=0x7fa0100fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:02:52.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- 172.21.15.76:0/3459169850 shutdown_connections 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- 172.21.15.76:0/3459169850 wait complete. 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 Processor -- start 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.886+0000 7fa016698700 1 -- start start 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa01006aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa010101910 0x7fa01019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f240 con 0x7fa010100710 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f3b0 con 0x7fa0101022b0 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01019f570 con 0x7fa010101910 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33260/0 (socket says 172.21.15.76:33260) 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 -- 172.21.15.76:0/851037941 learned_addr learned my addr 172.21.15.76:0/851037941 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00ffff700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa01006aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:52.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00f7fe700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa010101910 0x7fa01019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 -- 172.21.15.76:0/851037941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa010101910 msgr2=0x7fa01019c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa010101910 0x7fa01019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 -- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 msgr2=0x7fa01006aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa01006aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 -- 172.21.15.76:0/851037941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0040095f0 con 0x7fa0101022b0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00ffff700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa01006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa014c35700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b970 tx=0x7fa00800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00800c9b0 con 0x7fa0101022b0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01019f850 con 0x7fa0101022b0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa008014440 con 0x7fa0101022b0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.887+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080075d0 con 0x7fa0101022b0 2024-01-25T16:02:52.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.888+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01019fd50 con 0x7fa0101022b0 2024-01-25T16:02:52.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.888+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010107ad0 con 0x7fa0101022b0 2024-01-25T16:02:52.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.889+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa0080145b0 con 0x7fa0101022b0 2024-01-25T16:02:52.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.890+0000 7fa00d7fa700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 0x7f9ff80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:02:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.890+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fa008006e40 con 0x7fa0101022b0 2024-01-25T16:02:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.890+0000 7fa00ffff700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 0x7f9ff80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:02:52.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.891+0000 7fa00ffff700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 0x7f9ff80a38b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa010101770 tx=0x7fa000005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:02:52.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:52.894+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa0080956e0 con 0x7fa0101022b0 2024-01-25T16:02:53.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:02:53.103+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa01002d540 con 0x7f9ff80a13f0 2024-01-25T16:02:53.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:53 smithi076 ceph-mon[35642]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:53.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:53.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:53.706 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:53.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:53 smithi033 ceph-mon[31417]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:53.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:53.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:53.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:53.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:53 smithi152 ceph-mon[36015]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:02:53.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:02:53.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:02:53.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:02:54.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:54 smithi076 ceph-mon[35642]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:54.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:54 smithi033 ceph-mon[31417]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:54.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:54 smithi152 ceph-mon[36015]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:56.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:56 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/3979168090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:56 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:56 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]': finished 2024-01-25T16:02:56.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:56 smithi033 ceph-mon[31417]: osdmap e49: 8 total, 7 up, 8 in 2024-01-25T16:02:56.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:02:56.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:56 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/3979168090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:56 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:56 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]': finished 2024-01-25T16:02:56.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:56 smithi152 ceph-mon[36015]: osdmap e49: 8 total, 7 up, 8 in 2024-01-25T16:02:56.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:02:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:56 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/3979168090' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:56 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]: dispatch 2024-01-25T16:02:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:56 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dc7c7fb6-7719-4f21-8ef5-103d43fa1a49"}]': finished 2024-01-25T16:02:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:56 smithi076 ceph-mon[35642]: osdmap e49: 8 total, 7 up, 8 in 2024-01-25T16:02:56.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:02:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:57 smithi033 ceph-mon[31417]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:57 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/166974056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:57.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:57 smithi152 ceph-mon[36015]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:57.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:57 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/166974056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:57 smithi076 ceph-mon[35642]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:57 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/166974056' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:02:59.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:02:59 smithi152 ceph-mon[36015]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:02:59 smithi076 ceph-mon[35642]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:02:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:02:59 smithi033 ceph-mon[31417]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:01.222 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:01 smithi076 ceph-mon[35642]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:01 smithi033 ceph-mon[31417]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:01.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:01 smithi152 ceph-mon[36015]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:03.251 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:03 smithi076 ceph-mon[35642]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:03.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:03 smithi152 ceph-mon[36015]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:03 smithi033 ceph-mon[31417]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:04.146 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-25T16:03:04.146 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-25T16:03:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:04.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-01-25T16:03:04.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:05 smithi152 ceph-mon[36015]: Deploying daemon osd.7 on smithi076 2024-01-25T16:03:05.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:05 smithi152 ceph-mon[36015]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:05 smithi076 ceph-mon[35642]: Deploying daemon osd.7 on smithi076 2024-01-25T16:03:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:05 smithi076 ceph-mon[35642]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:05 smithi033 ceph-mon[31417]: Deploying daemon osd.7 on smithi076 2024-01-25T16:03:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:05 smithi033 ceph-mon[31417]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:07 smithi076 ceph-mon[35642]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:07 smithi033 ceph-mon[31417]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:07 smithi152 ceph-mon[36015]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:09.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:09 smithi076 ceph-mon[35642]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:09.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:09 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.407 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:09 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.407 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:09 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:09 smithi033 ceph-mon[31417]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:09.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:09 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:09 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:09 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:09 smithi152 ceph-mon[36015]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:09 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:09 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:09.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:09 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:11.104 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:11 smithi076 ceph-mon[35642]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:11 smithi033 ceph-mon[31417]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:11 smithi152 ceph-mon[36015]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:12.070 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 7 on host 'smithi076' 2024-01-25T16:03:12.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.067+0000 7fa00d7fa700 1 -- 172.21.15.76:0/851037941 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa01002d540 con 0x7f9ff80a13f0 2024-01-25T16:03:12.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 msgr2=0x7f9ff80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 0x7f9ff80a38b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa010101770 tx=0x7fa000005f90 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 msgr2=0x7fa01019eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa00800b970 tx=0x7fa00800bd30 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 shutdown_connections 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9ff80a13f0 0x7f9ff80a38b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0101022b0 0x7fa01019eaa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa010100710 0x7fa01006aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 --2- 172.21.15.76:0/851037941 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa010101910 0x7fa01019c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 >> 172.21.15.76:0/851037941 conn(0x7fa0100fbcd0 msgr2=0x7fa010104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 shutdown_connections 2024-01-25T16:03:12.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:03:12.069+0000 7fa016698700 1 -- 172.21.15.76:0/851037941 wait complete. 2024-01-25T16:03:12.673 DEBUG:teuthology.orchestra.run.smithi076:osd.7> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.7.service 2024-01-25T16:03:12.677 INFO:tasks.cephadm:Deploying osd.8 on smithi152 with /dev/vg_nvme/lv_2... 2024-01-25T16:03:12.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-01-25T16:03:12.921 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:03:13.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:13.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.309 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.309 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:13 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:13.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:13 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:13.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:13 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:14.216 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:14 smithi152 ceph-mon[36015]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-25T16:03:14.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:14 smithi152 ceph-mon[36015]: osdmap e50: 8 total, 7 up, 8 in 2024-01-25T16:03:14.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:14 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:14.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:14 smithi152 ceph-mon[36015]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.217 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:14 smithi152 ceph-mon[36015]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:14 smithi033 ceph-mon[31417]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-25T16:03:14.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:14 smithi033 ceph-mon[31417]: osdmap e50: 8 total, 7 up, 8 in 2024-01-25T16:03:14.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:14 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:14.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:14 smithi033 ceph-mon[31417]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:14 smithi033 ceph-mon[31417]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-mon[35642]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-01-25T16:03:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-mon[35642]: osdmap e50: 8 total, 7 up, 8 in 2024-01-25T16:03:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-mon[35642]: from='osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-mon[35642]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113907 s, 921 MB/s 2024-01-25T16:03:14.793 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-01-25T16:03:15.073 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:03:14.894+0000 7f91915c2700 -1 osd.7 0 waiting for initial osdmap 2024-01-25T16:03:15.073 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:03:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:03:14.913+0000 7f9189bb5700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:03:15.336 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: osdmap e51: 8 total, 7 up, 8 in 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: from='osd.7 ' entity='osd.7' 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] boot 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: osdmap e52: 8 total, 8 up, 8 in 2024-01-25T16:03:15.337 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.363 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_2 2024-01-25T16:03:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:03:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: osdmap e51: 8 total, 7 up, 8 in 2024-01-25T16:03:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: from='osd.7 ' entity='osd.7' 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] boot 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: osdmap e52: 8 total, 8 up, 8 in 2024-01-25T16:03:15.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:03:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: osdmap e51: 8 total, 7 up, 8 in 2024-01-25T16:03:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: from='osd.7 ' entity='osd.7' 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: osd.7 [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] boot 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: osdmap e52: 8 total, 8 up, 8 in 2024-01-25T16:03:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:03:15.605 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:03:16.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:16 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:03:16.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:16 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:03:16.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:16 smithi076 ceph-mon[35642]: osdmap e53: 8 total, 8 up, 8 in 2024-01-25T16:03:16.251 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:16 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:03:16.251 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:16 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:03:16.251 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:16 smithi152 ceph-mon[36015]: osdmap e53: 8 total, 8 up, 8 in 2024-01-25T16:03:16.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:16 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:03:16.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:16 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:03:16.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:16 smithi033 ceph-mon[31417]: osdmap e53: 8 total, 8 up, 8 in 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1102568125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8102e60 msgr2=0x7f07f8103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1102568125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8102e60 0x7f07f8103280 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f07e8009990 tx=0x7f07e8009ca0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1102568125 shutdown_connections 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1102568125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f81002a0 0x7f07f8109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1102568125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f8102e60 0x7f07f8103280 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1102568125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 0x7f07f80ffd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1102568125 >> 172.21.15.152:0/1102568125 conn(0x7f07f80fb480 msgr2=0x7f07f80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:03:17.044 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1102568125 shutdown_connections 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1102568125 wait complete. 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.039+0000 7f07fd5e7700 1 Processor -- start 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 -- start start 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 0x7f07f8102560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f8102e60 0x7f07f81010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f8101870 con 0x7f07f8102e60 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81019e0 con 0x7f07f80ff8e0 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07fd5e7700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f81a17f0 con 0x7f07f81002a0 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:47488/0 (socket says 172.21.15.152:47488) 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 -- 172.21.15.152:0/1619469302 learned_addr learned my addr 172.21.15.152:0/1619469302 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07f6ffd700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 0x7f07f8102560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:17.045 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07f77fe700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f8102e60 0x7f07f81010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 -- 172.21.15.152:0/1619469302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 msgr2=0x7f07f8102560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 0x7f07f8102560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 -- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f8102e60 msgr2=0x7f07f81010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f8102e60 0x7f07f81010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 -- 172.21.15.152:0/1619469302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80095f0 con 0x7f07f81002a0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07ee7fc700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b500 tx=0x7f07e000b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.040+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0011830 con 0x7f07f81002a0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.041+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f81a1ad0 con 0x7f07f81002a0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.041+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07e0011e50 con 0x7f07f81002a0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.041+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07e0010dc0 con 0x7f07f81002a0 2024-01-25T16:03:17.046 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.041+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f81a2020 con 0x7f07f81002a0 2024-01-25T16:03:17.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.041+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f8108dc0 con 0x7f07f81002a0 2024-01-25T16:03:17.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.043+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f07e0010380 con 0x7f07f81002a0 2024-01-25T16:03:17.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.044+0000 7f07f4ff9700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 0x7f07e40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:17.047 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.044+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f07e00cbf90 con 0x7f07f81002a0 2024-01-25T16:03:17.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.044+0000 7f07f6ffd700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 0x7f07e40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:17.048 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.045+0000 7f07f6ffd700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 0x7f07e40a3830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b5c0 tx=0x7f07e8003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:03:17.050 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.047+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f07e0095f50 con 0x7f07f81002a0 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:17 smithi152 ceph-mon[36015]: osdmap e54: 8 total, 8 up, 8 in 2024-01-25T16:03:17.252 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:17.249+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f07f802d5a0 con 0x7f07e40a1370 2024-01-25T16:03:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:17 smithi076 ceph-mon[35642]: osdmap e54: 8 total, 8 up, 8 in 2024-01-25T16:03:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:17.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:17 smithi033 ceph-mon[31417]: osdmap e54: 8 total, 8 up, 8 in 2024-01-25T16:03:18.125 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:18 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi076 2024-01-25T16:03:18.125 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:18 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 5693M 2024-01-25T16:03:18.125 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:18.125 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:18.125 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:18 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi076 2024-01-25T16:03:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:18 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 5693M 2024-01-25T16:03:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:18.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:18 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi076 2024-01-25T16:03:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:18 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 5693M 2024-01-25T16:03:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:18.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:18.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:19.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:19 smithi152 ceph-mon[36015]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:19.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:19 smithi152 ceph-mon[36015]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:19.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:19 smithi152 ceph-mon[36015]: osdmap e55: 8 total, 8 up, 8 in 2024-01-25T16:03:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:19 smithi076 ceph-mon[35642]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:19 smithi076 ceph-mon[35642]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:19 smithi076 ceph-mon[35642]: osdmap e55: 8 total, 8 up, 8 in 2024-01-25T16:03:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:19 smithi033 ceph-mon[31417]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:19 smithi033 ceph-mon[31417]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:19 smithi033 ceph-mon[31417]: osdmap e55: 8 total, 8 up, 8 in 2024-01-25T16:03:20.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: osdmap e56: 8 total, 8 up, 8 in 2024-01-25T16:03:20.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/3147499203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]': finished 2024-01-25T16:03:20.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: osdmap e57: 9 total, 8 up, 9 in 2024-01-25T16:03:20.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:20.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: osdmap e56: 8 total, 8 up, 8 in 2024-01-25T16:03:20.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/3147499203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]': finished 2024-01-25T16:03:20.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: osdmap e57: 9 total, 8 up, 9 in 2024-01-25T16:03:20.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:20.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: osdmap e56: 8 total, 8 up, 8 in 2024-01-25T16:03:20.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/3147499203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]: dispatch 2024-01-25T16:03:20.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aedc05dd-ab90-4ad9-9937-a3a7b56d4f72"}]': finished 2024-01-25T16:03:20.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: osdmap e57: 9 total, 8 up, 9 in 2024-01-25T16:03:20.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:21.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:21 smithi152 ceph-mon[36015]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:21.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:21 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/3013825552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:21 smithi076 ceph-mon[35642]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:21 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/3013825552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:21 smithi033 ceph-mon[31417]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:21 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/3013825552' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:23.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:23 smithi152 ceph-mon[36015]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:23.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:23 smithi076 ceph-mon[35642]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:23 smithi033 ceph-mon[31417]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 241 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:25.308 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:25 smithi152 ceph-mon[36015]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:25 smithi076 ceph-mon[35642]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:25 smithi033 ceph-mon[31417]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:27.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:27 smithi152 ceph-mon[36015]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:27 smithi076 ceph-mon[35642]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:27 smithi033 ceph-mon[31417]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:28.198 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-25T16:03:28.198 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:28.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-25T16:03:28.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:28.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-01-25T16:03:28.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:29.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:29 smithi152 ceph-mon[36015]: Deploying daemon osd.8 on smithi152 2024-01-25T16:03:29.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:29 smithi152 ceph-mon[36015]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:29 smithi076 ceph-mon[35642]: Deploying daemon osd.8 on smithi152 2024-01-25T16:03:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:29 smithi076 ceph-mon[35642]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:29 smithi033 ceph-mon[31417]: Deploying daemon osd.8 on smithi152 2024-01-25T16:03:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:29 smithi033 ceph-mon[31417]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:31.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:31 smithi152 ceph-mon[36015]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:31 smithi033 ceph-mon[31417]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:31 smithi076 ceph-mon[35642]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:33.355 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:33 smithi152 ceph-mon[36015]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:33.355 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:33 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.355 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:33 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.355 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:33 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:33 smithi076 ceph-mon[35642]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:33 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:33 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:33 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:33 smithi033 ceph-mon[31417]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:33 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:33 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:33.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:33 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:35 smithi152 ceph-mon[36015]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:35 smithi076 ceph-mon[35642]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:35 smithi033 ceph-mon[31417]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:35.774 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 8 on host 'smithi152' 2024-01-25T16:03:35.774 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.771+0000 7f07f4ff9700 1 -- 172.21.15.152:0/1619469302 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f07f802d5a0 con 0x7f07e40a1370 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.774+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 msgr2=0x7f07e40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.774+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 0x7f07e40a3830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f07e800b5c0 tx=0x7f07e8003790 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 msgr2=0x7f07f8100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b500 tx=0x7f07e000b810 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 shutdown_connections 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07e40a1370 0x7f07e40a3830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07f8102e60 0x7f07f81010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07f80ff8e0 0x7f07f8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 --2- 172.21.15.152:0/1619469302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f07f81002a0 0x7f07f8100bb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 >> 172.21.15.152:0/1619469302 conn(0x7f07f80fb480 msgr2=0x7f07f8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:03:35.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 shutdown_connections 2024-01-25T16:03:35.779 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:03:35.775+0000 7f07fd5e7700 1 -- 172.21.15.152:0/1619469302 wait complete. 2024-01-25T16:03:36.373 DEBUG:teuthology.orchestra.run.smithi152:osd.8> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.8.service 2024-01-25T16:03:36.375 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-01-25T16:03:36.375 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-25T16:03:36.633 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:03:36.457+0000 7f8c839607c0 -1 osd.8 0 log_to_monitors true 2024-01-25T16:03:36.641 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:03:37.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.073 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.074 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:37.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-01-25T16:03:37.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:37.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:38.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:37 smithi033 ceph-mon[31417]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-25T16:03:38.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:37 smithi033 ceph-mon[31417]: osdmap e58: 9 total, 8 up, 9 in 2024-01-25T16:03:38.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:38.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:37 smithi033 ceph-mon[31417]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:37 smithi033 ceph-mon[31417]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.124 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:37 smithi152 ceph-mon[36015]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-25T16:03:38.124 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:37 smithi152 ceph-mon[36015]: osdmap e58: 9 total, 8 up, 9 in 2024-01-25T16:03:38.124 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:38.124 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:37 smithi152 ceph-mon[36015]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.124 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:37 smithi152 ceph-mon[36015]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:37 smithi076 ceph-mon[35642]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-01-25T16:03:38.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:37 smithi076 ceph-mon[35642]: osdmap e58: 9 total, 8 up, 9 in 2024-01-25T16:03:38.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:38.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:37 smithi076 ceph-mon[35642]: from='osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:37 smithi076 ceph-mon[35642]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:03:38.633 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:03:38.617+0000 7f8c79fc8700 -1 osd.8 0 waiting for initial osdmap 2024-01-25T16:03:38.799 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-25T16:03:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-25T16:03:38.800 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-01-25T16:03:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-01-25T16:03:38.800 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011545 s, 908 MB/s 2024-01-25T16:03:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-01-25T16:03:39.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:03:39.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: osdmap e59: 9 total, 8 up, 9 in 2024-01-25T16:03:39.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:39.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:38 smithi033 ceph-mon[31417]: from='osd.8 ' entity='osd.8' 2024-01-25T16:03:39.085 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:03:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: osdmap e59: 9 total, 8 up, 9 in 2024-01-25T16:03:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:39.086 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-mon[36015]: from='osd.8 ' entity='osd.8' 2024-01-25T16:03:39.086 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:03:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:03:38.633+0000 7f8c70db8700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:03:39.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:03:39.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: osdmap e59: 9 total, 8 up, 9 in 2024-01-25T16:03:39.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:39.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 715 GiB / 715 GiB avail 2024-01-25T16:03:39.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:38 smithi076 ceph-mon[35642]: from='osd.8 ' entity='osd.8' 2024-01-25T16:03:39.705 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-01-25T16:03:39.956 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:03:40.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] boot 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: osdmap e60: 9 total, 9 up, 9 in 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:40.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:40.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: osdmap e61: 9 total, 9 up, 9 in 2024-01-25T16:03:40.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:39 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:03:40.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:03:40.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] boot 2024-01-25T16:03:40.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: osdmap e60: 9 total, 9 up, 9 in 2024-01-25T16:03:40.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: osdmap e61: 9 total, 9 up, 9 in 2024-01-25T16:03:40.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:39 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: osd.8 [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] boot 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: osdmap e60: 9 total, 9 up, 9 in 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:03:40.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:40.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:40.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: osdmap e61: 9 total, 9 up, 9 in 2024-01-25T16:03:40.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:39 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi152 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 5693M 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:40.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:40 smithi076 ceph-mon[35642]: osdmap e62: 9 total, 9 up, 9 in 2024-01-25T16:03:41.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi152 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 5693M 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:41.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:40 smithi033 ceph-mon[31417]: osdmap e62: 9 total, 9 up, 9 in 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi152 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 5693M 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:41.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:40 smithi152 ceph-mon[36015]: osdmap e62: 9 total, 9 up, 9 in 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 -- 172.21.15.33:0/1837340575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 msgr2=0x7efdc4100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1837340575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4100b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7efdbc00b0d0 tx=0x7efdbc00b3e0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 -- 172.21.15.33:0/1837340575 shutdown_connections 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1837340575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc41022a0 0x7efdc4108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1837340575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4100b20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1837340575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc4101900 0x7efdc4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 -- 172.21.15.33:0/1837340575 >> 172.21.15.33:0/1837340575 conn(0x7efdc40fbca0 msgr2=0x7efdc40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:03:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 -- 172.21.15.33:0/1837340575 shutdown_connections 2024-01-25T16:03:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.566+0000 7efdc96b4700 1 -- 172.21.15.33:0/1837340575 wait complete. 2024-01-25T16:03:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 Processor -- start 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 -- start start 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc4101900 0x7efdc4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc41022a0 0x7efdc419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419f390 con 0x7efdc4100700 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419f4d0 con 0x7efdc41022a0 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc96b4700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc419f660 con 0x7efdc4101900 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43518/0 (socket says 172.21.15.33:43518) 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 -- 172.21.15.33:0/1762660552 learned_addr learned my addr 172.21.15.33:0/1762660552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:03:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc27fc700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc4101900 0x7efdc4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc37fe700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc41022a0 0x7efdc419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 -- 172.21.15.33:0/1762660552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc4101900 msgr2=0x7efdc4197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc4101900 0x7efdc4197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 -- 172.21.15.33:0/1762660552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc41022a0 msgr2=0x7efdc419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.567+0000 7efdc2ffd700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc41022a0 0x7efdc419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdc2ffd700 1 -- 172.21.15.33:0/1762660552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009d00 con 0x7efdc4100700 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdc2ffd700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0094c0 tx=0x7efdbc0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00e040 con 0x7efdc4100700 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc419f8e0 con 0x7efdc4100700 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdbc01dd50 con 0x7efdc4100700 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc01b450 con 0x7efdc4100700 2024-01-25T16:03:41.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.568+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc419fd50 con 0x7efdc4100700 2024-01-25T16:03:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.569+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc40684a0 con 0x7efdc4100700 2024-01-25T16:03:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.570+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7efdbc01d890 con 0x7efdc4100700 2024-01-25T16:03:41.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.571+0000 7efdbbfff700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 0x7efdb00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:03:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.571+0000 7efdc27fc700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 0x7efdb00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:03:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.571+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7efdbc0ce350 con 0x7efdc4100700 2024-01-25T16:03:41.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.571+0000 7efdc27fc700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 0x7efdb00a3830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efdac007c60 tx=0x7efdac0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:03:41.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.575+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7efdbc098310 con 0x7efdc4100700 2024-01-25T16:03:41.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:03:41.785+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efdc40fdea0 con 0x7efdb00a1370 2024-01-25T16:03:43.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: osdmap e63: 9 total, 9 up, 9 in 2024-01-25T16:03:43.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:43.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:43.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:43.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:43.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:42 smithi033 ceph-mon[31417]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: osdmap e63: 9 total, 9 up, 9 in 2024-01-25T16:03:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:42 smithi152 ceph-mon[36015]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:43.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: osdmap e63: 9 total, 9 up, 9 in 2024-01-25T16:03:43.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: from='client.14433 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:03:43.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:03:43.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:03:43.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:43.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:42 smithi076 ceph-mon[35642]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 673 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:44.023 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:43 smithi033 ceph-mon[31417]: osdmap e64: 9 total, 9 up, 9 in 2024-01-25T16:03:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:43 smithi152 ceph-mon[36015]: osdmap e64: 9 total, 9 up, 9 in 2024-01-25T16:03:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:43 smithi076 ceph-mon[35642]: osdmap e64: 9 total, 9 up, 9 in 2024-01-25T16:03:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:44 smithi033 ceph-mon[31417]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]: dispatch 2024-01-25T16:03:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]': finished 2024-01-25T16:03:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:44 smithi033 ceph-mon[31417]: osdmap e65: 10 total, 9 up, 10 in 2024-01-25T16:03:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:03:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:44 smithi152 ceph-mon[36015]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]: dispatch 2024-01-25T16:03:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]': finished 2024-01-25T16:03:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:44 smithi152 ceph-mon[36015]: osdmap e65: 10 total, 9 up, 10 in 2024-01-25T16:03:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:03:45.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:44 smithi076 ceph-mon[35642]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:45.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]: dispatch 2024-01-25T16:03:45.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3063976346' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57cd9e56-03a7-4be8-98c0-27b71dadb5a7"}]': finished 2024-01-25T16:03:45.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:44 smithi076 ceph-mon[35642]: osdmap e65: 10 total, 9 up, 10 in 2024-01-25T16:03:45.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:03:46.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:46 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2759292704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:46.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:46 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2759292704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:46.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:46 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2759292704' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:03:47.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:47 smithi033 ceph-mon[31417]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-25T16:03:47.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:47 smithi152 ceph-mon[36015]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-25T16:03:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:47 smithi076 ceph-mon[35642]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-01-25T16:03:49.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:49 smithi033 ceph-mon[31417]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-25T16:03:49.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:49 smithi152 ceph-mon[36015]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-25T16:03:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:49 smithi076 ceph-mon[35642]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-01-25T16:03:51.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:51 smithi033 ceph-mon[31417]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:03:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:51 smithi152 ceph-mon[36015]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:03:51.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:51 smithi076 ceph-mon[35642]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:03:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:53 smithi152 ceph-mon[36015]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-25T16:03:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-25T16:03:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:53 smithi076 ceph-mon[35642]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-25T16:03:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-25T16:03:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:53.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:53 smithi033 ceph-mon[31417]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-01-25T16:03:53.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-01-25T16:03:53.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:03:54.266 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:54 smithi033 ceph-mon[31417]: Deploying daemon osd.9 on smithi033 2024-01-25T16:03:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:54 smithi076 ceph-mon[35642]: Deploying daemon osd.9 on smithi033 2024-01-25T16:03:54.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:54 smithi152 ceph-mon[36015]: Deploying daemon osd.9 on smithi033 2024-01-25T16:03:55.230 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:55 smithi033 ceph-mon[31417]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:03:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:55 smithi076 ceph-mon[35642]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:03:55.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:55 smithi152 ceph-mon[36015]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:03:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:57 smithi033 ceph-mon[31417]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:03:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:57 smithi076 ceph-mon[35642]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:03:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:57 smithi152 ceph-mon[36015]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-01-25T16:03:59.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:59 smithi033 ceph-mon[31417]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:59.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:59 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:59 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.278 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:03:59 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:59 smithi076 ceph-mon[35642]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:59 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:59 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:03:59 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:03:59.436 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:59 smithi152 ceph-mon[36015]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:03:59.436 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:59 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.436 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:59 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:03:59.436 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:03:59 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:01 smithi033 ceph-mon[31417]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:01.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:01 smithi152 ceph-mon[36015]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:01 smithi076 ceph-mon[35642]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:02.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.196+0000 7efdbbfff700 1 -- 172.21.15.33:0/1762660552 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efdc40fdea0 con 0x7efdb00a1370 2024-01-25T16:04:02.199 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 msgr2=0x7efdb00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 0x7efdb00a3830 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efdac007c60 tx=0x7efdac0073d0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 msgr2=0x7efdc4197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efdbc0094c0 tx=0x7efdbc0094f0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 shutdown_connections 2024-01-25T16:04:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7efdb00a1370 0x7efdb00a3830 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc41022a0 0x7efdc419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdc4100700 0x7efdc4197850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 --2- 172.21.15.33:0/1762660552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efdc4101900 0x7efdc4197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.198+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 >> 172.21.15.33:0/1762660552 conn(0x7efdc40fbca0 msgr2=0x7efdc40fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.199+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 shutdown_connections 2024-01-25T16:04:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:02.199+0000 7efdc96b4700 1 -- 172.21.15.33:0/1762660552 wait complete. 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:02.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:02 smithi076 ceph-mon[35642]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-25T16:04:02.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:02.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:02 smithi033 ceph-mon[31417]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-25T16:04:02.877 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.9.service 2024-01-25T16:04:02.879 INFO:tasks.cephadm:Deploying osd.10 on smithi076 with /dev/vg_nvme/lv_1... 2024-01-25T16:04:02.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:02.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:02.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:02.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:02 smithi152 ceph-mon[36015]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-01-25T16:04:03.124 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:04:04.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-mon[31417]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-25T16:04:04.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-mon[31417]: osdmap e66: 10 total, 9 up, 10 in 2024-01-25T16:04:04.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-mon[31417]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:04:04.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:04.349 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:04:04.188+0000 7f225a5e0700 -1 osd.9 0 waiting for initial osdmap 2024-01-25T16:04:04.349 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:04:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:04:04.203+0000 7f2256bdb700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:04:04.492 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:04 smithi076 ceph-mon[35642]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-25T16:04:04.492 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:04 smithi076 ceph-mon[35642]: osdmap e66: 10 total, 9 up, 10 in 2024-01-25T16:04:04.492 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:04 smithi076 ceph-mon[35642]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:04:04.492 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:04 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:04.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:04 smithi152 ceph-mon[36015]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-01-25T16:04:04.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:04 smithi152 ceph-mon[36015]: osdmap e66: 10 total, 9 up, 10 in 2024-01-25T16:04:04.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:04 smithi152 ceph-mon[36015]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-01-25T16:04:04.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:04 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.053 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-25T16:04:05.053 INFO:teuthology.orchestra.run.smithi076.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-25T16:04:05.053 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10+0 records in 2024-01-25T16:04:05.053 INFO:teuthology.orchestra.run.smithi076.stderr:10+0 records out 2024-01-25T16:04:05.053 INFO:teuthology.orchestra.run.smithi076.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112872 s, 929 MB/s 2024-01-25T16:04:05.054 INFO:teuthology.orchestra.run.smithi076.stderr:--> Zapping successful for: 2024-01-25T16:04:05.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:05 smithi033 ceph-mon[31417]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:05.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:05 smithi033 ceph-mon[31417]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:04:05.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:05 smithi033 ceph-mon[31417]: osdmap e67: 10 total, 9 up, 10 in 2024-01-25T16:04:05.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:05 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.470 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:05 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.510 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:05 smithi076 ceph-mon[35642]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:05.510 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:05 smithi076 ceph-mon[35642]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:04:05.510 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:05 smithi076 ceph-mon[35642]: osdmap e67: 10 total, 9 up, 10 in 2024-01-25T16:04:05.510 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:05 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.510 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:05 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:05 smithi152 ceph-mon[36015]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:05 smithi152 ceph-mon[36015]: from='osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-01-25T16:04:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:05 smithi152 ceph-mon[36015]: osdmap e67: 10 total, 9 up, 10 in 2024-01-25T16:04:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:05 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:05 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:05.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi076:vg_nvme/lv_1 2024-01-25T16:04:06.032 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:04:06.405 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:04:06.405 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:04:06.405 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] boot 2024-01-25T16:04:06.405 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: osdmap e68: 10 total, 10 up, 10 in 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:06.406 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:06 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] boot 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: osdmap e68: 10 total, 10 up, 10 in 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:06.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:06.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:06 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: osd.9 [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] boot 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: osdmap e68: 10 total, 10 up, 10 in 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:06.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:06 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:07 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi033 2024-01-25T16:04:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:07 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 4273M 2024-01-25T16:04:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:07 smithi076 ceph-mon[35642]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:07 smithi076 ceph-mon[35642]: osdmap e69: 10 total, 10 up, 10 in 2024-01-25T16:04:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:07 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi033 2024-01-25T16:04:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:07 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 4273M 2024-01-25T16:04:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:07 smithi033 ceph-mon[31417]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:07 smithi033 ceph-mon[31417]: osdmap e69: 10 total, 10 up, 10 in 2024-01-25T16:04:07.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.624+0000 7fce42133700 1 -- 172.21.15.76:0/4028829340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 msgr2=0x7fce3c100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:07.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.624+0000 7fce42133700 1 --2- 172.21.15.76:0/4028829340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c100e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fce30009990 tx=0x7fce30009ca0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 -- 172.21.15.76:0/4028829340 shutdown_connections 2024-01-25T16:04:07.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 --2- 172.21.15.76:0/4028829340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c1013d0 0x7fce3c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 --2- 172.21.15.76:0/4028829340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c101cf0 0x7fce3c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 --2- 172.21.15.76:0/4028829340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c100e90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 -- 172.21.15.76:0/4028829340 >> 172.21.15.76:0/4028829340 conn(0x7fce3c0fb480 msgr2=0x7fce3c0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 -- 172.21.15.76:0/4028829340 shutdown_connections 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 -- 172.21.15.76:0/4028829340 wait complete. 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 Processor -- start 2024-01-25T16:04:07.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.625+0000 7fce42133700 1 -- start start 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c1013d0 0x7fce3c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c101cf0 0x7fce3c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f190 con 0x7fce3c101cf0 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f2d0 con 0x7fce3c100a30 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c19f460 con 0x7fce3c1013d0 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36078/0 (socket says 172.21.15.76:36078) 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 -- 172.21.15.76:0/3166809295 learned_addr learned my addr 172.21.15.76:0/3166809295 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3affd700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c1013d0 0x7fce3c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3bfff700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c101cf0 0x7fce3c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 -- 172.21.15.76:0/3166809295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c1013d0 msgr2=0x7fce3c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:07.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c1013d0 0x7fce3c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 -- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c101cf0 msgr2=0x7fce3c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c101cf0 0x7fce3c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 -- 172.21.15.76:0/3166809295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300095f0 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce3b7fe700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c990 tx=0x7fce2c00cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c007b00 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce2c007c60 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c19f740 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c018660 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.626+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c19fc40 con 0x7fce3c100a30 2024-01-25T16:04:07.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.627+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c107b30 con 0x7fce3c100a30 2024-01-25T16:04:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:07 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi033 2024-01-25T16:04:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:07 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 4273M 2024-01-25T16:04:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:07 smithi152 ceph-mon[36015]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-01-25T16:04:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:07 smithi152 ceph-mon[36015]: osdmap e69: 10 total, 10 up, 10 in 2024-01-25T16:04:07.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.628+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fce2c00f450 con 0x7fce3c100a30 2024-01-25T16:04:07.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.629+0000 7fce38ff9700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 0x7fce240a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:07.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.629+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fce2c0cc320 con 0x7fce3c100a30 2024-01-25T16:04:07.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.629+0000 7fce3affd700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 0x7fce240a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:07.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.630+0000 7fce3affd700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 0x7fce240a3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b5c0 tx=0x7fce30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:07.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.633+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fce2c095fa0 con 0x7fce3c100a30 2024-01-25T16:04:07.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:07.833+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fce3c02d5e0 con 0x7fce240a1370 2024-01-25T16:04:08.469 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:08.469 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:08.469 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:08.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:08.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:08.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:08.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:08.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:08.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:09.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:09 smithi033 ceph-mon[31417]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:09.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:09 smithi033 ceph-mon[31417]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:09.484 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:09 smithi076 ceph-mon[35642]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:09.484 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:09 smithi076 ceph-mon[35642]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:09 smithi152 ceph-mon[36015]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi076:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:09 smithi152 ceph-mon[36015]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/1846558393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]': finished 2024-01-25T16:04:11.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: osdmap e70: 11 total, 10 up, 11 in 2024-01-25T16:04:11.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:11 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/1846558393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]': finished 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: osdmap e70: 11 total, 10 up, 11 in 2024-01-25T16:04:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:11 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/1846558393' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]: dispatch 2024-01-25T16:04:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06adc495-98fd-4870-97cf-0cd51993ca94"}]': finished 2024-01-25T16:04:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: osdmap e70: 11 total, 10 up, 11 in 2024-01-25T16:04:11.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:11 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:12.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.76:0/3406953572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:12.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:12 smithi152 ceph-mon[36015]: from='client.? 172.21.15.76:0/3406953572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:12.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:12 smithi076 ceph-mon[35642]: from='client.? 172.21.15.76:0/3406953572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:13.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:13 smithi033 ceph-mon[31417]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:13.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:13 smithi152 ceph-mon[36015]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:13 smithi076 ceph-mon[35642]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:15 smithi076 ceph-mon[35642]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:15 smithi033 ceph-mon[31417]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:15.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:15 smithi152 ceph-mon[36015]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:17 smithi076 ceph-mon[35642]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:17 smithi033 ceph-mon[31417]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:17 smithi152 ceph-mon[36015]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:19 smithi076 ceph-mon[35642]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:19 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-25T16:04:19.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:19 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:19.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:19 smithi076 ceph-mon[35642]: Deploying daemon osd.10 on smithi076 2024-01-25T16:04:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:19 smithi033 ceph-mon[31417]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:19 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-25T16:04:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:19 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:19 smithi033 ceph-mon[31417]: Deploying daemon osd.10 on smithi076 2024-01-25T16:04:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:19 smithi152 ceph-mon[36015]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:19 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-01-25T16:04:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:19 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:19 smithi152 ceph-mon[36015]: Deploying daemon osd.10 on smithi076 2024-01-25T16:04:21.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:21 smithi076 ceph-mon[35642]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:21.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:21 smithi033 ceph-mon[31417]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:21.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:21 smithi152 ceph-mon[36015]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:23.356 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:23 smithi076 ceph-mon[35642]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:23 smithi033 ceph-mon[31417]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:23 smithi152 ceph-mon[36015]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:24.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:24.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:24.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:24 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:24 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:24.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:24.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:24.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:24 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:25 smithi076 ceph-mon[35642]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:25 smithi033 ceph-mon[31417]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:25.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:25 smithi152 ceph-mon[36015]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:27.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:27.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:27 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:Created osd(s) 10 on host 'smithi076' 2024-01-25T16:04:27.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.551+0000 7fce38ff9700 1 -- 172.21.15.76:0/3166809295 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fce3c02d5e0 con 0x7fce240a1370 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 msgr2=0x7fce240a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 0x7fce240a3830 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fce3000b5c0 tx=0x7fce30005fb0 comp rx=0 tx=0).stop 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 msgr2=0x7fce3c197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c990 tx=0x7fce2c00cd50 comp rx=0 tx=0).stop 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 shutdown_connections 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fce240a1370 0x7fce240a3830 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce3c101cf0 0x7fce3c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:27.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce3c100a30 0x7fce3c197600 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:27.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 --2- 172.21.15.76:0/3166809295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fce3c1013d0 0x7fce3c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:27.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 >> 172.21.15.76:0/3166809295 conn(0x7fce3c0fb480 msgr2=0x7fce3c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:27.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 shutdown_connections 2024-01-25T16:04:27.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-01-25T16:04:27.553+0000 7fce42133700 1 -- 172.21.15.76:0/3166809295 wait complete. 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:27 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:27 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.211 DEBUG:teuthology.orchestra.run.smithi076:osd.10> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.10.service 2024-01-25T16:04:28.213 INFO:tasks.cephadm:Deploying osd.11 on smithi152 with /dev/vg_nvme/lv_1... 2024-01-25T16:04:28.213 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-01-25T16:04:28.459 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.793 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:28 smithi152 ceph-mon[36015]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:28.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:28.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:28 smithi033 ceph-mon[31417]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:28.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:28.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:28.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-01-25T16:04:28.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:28 smithi076 ceph-mon[35642]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:29.721 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-mon[35642]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-25T16:04:29.721 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-mon[35642]: osdmap e71: 11 total, 10 up, 11 in 2024-01-25T16:04:29.722 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:29.722 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-mon[35642]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:29.722 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-mon[35642]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:29.722 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:04:29.544+0000 7f74b907e700 -1 osd.10 0 waiting for initial osdmap 2024-01-25T16:04:29.722 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:04:29 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:04:29.557+0000 7f74b5679700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:04:29.787 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:29 smithi152 ceph-mon[36015]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-25T16:04:29.788 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:29 smithi152 ceph-mon[36015]: osdmap e71: 11 total, 10 up, 11 in 2024-01-25T16:04:29.788 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:29 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:29.788 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:29 smithi152 ceph-mon[36015]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:29.788 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:29 smithi152 ceph-mon[36015]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:29.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:29 smithi033 ceph-mon[31417]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-01-25T16:04:29.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:29 smithi033 ceph-mon[31417]: osdmap e71: 11 total, 10 up, 11 in 2024-01-25T16:04:29.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:29 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:29.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:29 smithi033 ceph-mon[31417]: from='osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:29.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:29 smithi033 ceph-mon[31417]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]: dispatch 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10+0 records in 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr:10+0 records out 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114844 s, 913 MB/s 2024-01-25T16:04:30.305 INFO:teuthology.orchestra.run.smithi152.stderr:--> Zapping successful for: 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: osdmap e72: 11 total, 10 up, 11 in 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] boot 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: osdmap e73: 11 total, 11 up, 11 in 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:30 smithi076 ceph-mon[35642]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:30.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: osdmap e72: 11 total, 10 up, 11 in 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] boot 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: osdmap e73: 11 total, 11 up, 11 in 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.816 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:30 smithi033 ceph-mon[31417]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi076", "root=default"]}]': finished 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: osdmap e72: 11 total, 10 up, 11 in 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: osd.10 [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] boot 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: osdmap e73: 11 total, 11 up, 11 in 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:04:30.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:30 smithi152 ceph-mon[36015]: pgmap v234: 1 pgs: 1 remapped+peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-01-25T16:04:31.016 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch daemon add osd smithi152:vg_nvme/lv_1 2024-01-25T16:04:31.261 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:04:31.675 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:04:31.675 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:04:31.675 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: osdmap e74: 11 total, 11 up, 11 in 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi076 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 4270M 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:31.676 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:04:31.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: osdmap e74: 11 total, 11 up, 11 in 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi076 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 4270M 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:31.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: osdmap e74: 11 total, 11 up, 11 in 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi076 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 4270M 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:31.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:32.863 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 -- 172.21.15.152:0/3687988979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 msgr2=0x7fb04c108bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3687988979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c108bb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb04400b0a0 tx=0x7fb04400b3b0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 -- 172.21.15.152:0/3687988979 shutdown_connections 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3687988979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c108bb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3687988979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 0x7fb04c100b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3687988979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 -- 172.21.15.152:0/3687988979 >> 172.21.15.152:0/3687988979 conn(0x7fb04c0fbce0 msgr2=0x7fb04c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 -- 172.21.15.152:0/3687988979 shutdown_connections 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.859+0000 7fb050aa2700 1 -- 172.21.15.152:0/3687988979 wait complete. 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 Processor -- start 2024-01-25T16:04:32.864 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 -- start start 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 0x7fb04c196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c19e1d0 con 0x7fb04c101960 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c19e310 con 0x7fb04c100760 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb050aa2700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c19e4a0 con 0x7fb04c102300 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50866/0 (socket says 172.21.15.152:50866) 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 -- 172.21.15.152:0/3621127020 learned_addr learned my addr 172.21.15.152:0/3621127020 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04a59c700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 0x7fb04c196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:32.865 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb049d9b700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 -- 172.21.15.152:0/3621127020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 msgr2=0x7fb04c196640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 0x7fb04c196640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 -- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 msgr2=0x7fb04c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb04ad9d700 1 -- 172.21.15.152:0/3621127020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044009d00 con 0x7fb04c102300 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.860+0000 7fb049d9b700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb04ad9d700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb044004d30 tx=0x7fb044004d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400e040 con 0x7fb04c102300 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb04400ce50 con 0x7fb04c102300 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c19e720 con 0x7fb04c102300 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04401bb60 con 0x7fb04c102300 2024-01-25T16:04:32.866 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.861+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c19eb40 con 0x7fb04c102300 2024-01-25T16:04:32.867 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.862+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c107a50 con 0x7fb04c102300 2024-01-25T16:04:32.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.863+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb04401bcc0 con 0x7fb04c102300 2024-01-25T16:04:32.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.863+0000 7fb03b7fe700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 0x7fb0340a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:32.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.863+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fb0440ce3e0 con 0x7fb04c102300 2024-01-25T16:04:32.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.864+0000 7fb04a59c700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 0x7fb0340a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:32.868 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.864+0000 7fb04a59c700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 0x7fb0340a3830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb04c1017c0 tx=0x7fb03c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:32.870 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:32.868+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb044097f20 con 0x7fb04c102300 2024-01-25T16:04:33.069 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:33.066+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb04c02d540 con 0x7fb0340a1370 2024-01-25T16:04:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:33 smithi033 ceph-mon[31417]: osdmap e75: 11 total, 11 up, 11 in 2024-01-25T16:04:33.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:33 smithi033 ceph-mon[31417]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:33.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:33 smithi152 ceph-mon[36015]: osdmap e75: 11 total, 11 up, 11 in 2024-01-25T16:04:33.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:33 smithi152 ceph-mon[36015]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:33 smithi076 ceph-mon[35642]: osdmap e75: 11 total, 11 up, 11 in 2024-01-25T16:04:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:33 smithi076 ceph-mon[35642]: pgmap v237: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:34 smithi033 ceph-mon[31417]: osdmap e76: 11 total, 11 up, 11 in 2024-01-25T16:04:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:34 smithi033 ceph-mon[31417]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:34.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:34 smithi152 ceph-mon[36015]: osdmap e76: 11 total, 11 up, 11 in 2024-01-25T16:04:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:34 smithi152 ceph-mon[36015]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:34 smithi076 ceph-mon[35642]: osdmap e76: 11 total, 11 up, 11 in 2024-01-25T16:04:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:34 smithi076 ceph-mon[35642]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi152:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:04:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-01-25T16:04:34.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-01-25T16:04:34.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:35.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:35 smithi033 ceph-mon[31417]: osdmap e77: 11 total, 11 up, 11 in 2024-01-25T16:04:35.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:35 smithi033 ceph-mon[31417]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:35 smithi152 ceph-mon[36015]: osdmap e77: 11 total, 11 up, 11 in 2024-01-25T16:04:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:35 smithi152 ceph-mon[36015]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:35 smithi076 ceph-mon[35642]: osdmap e77: 11 total, 11 up, 11 in 2024-01-25T16:04:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:35 smithi076 ceph-mon[35642]: pgmap v240: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:36.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/4131291849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:36 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:36 smithi033 ceph-mon[31417]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]': finished 2024-01-25T16:04:36.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:36 smithi033 ceph-mon[31417]: osdmap e78: 12 total, 11 up, 12 in 2024-01-25T16:04:36.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:36 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/4131291849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:36 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:36 smithi152 ceph-mon[36015]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]': finished 2024-01-25T16:04:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:36 smithi152 ceph-mon[36015]: osdmap e78: 12 total, 11 up, 12 in 2024-01-25T16:04:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:36 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:36.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/4131291849' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:36 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]: dispatch 2024-01-25T16:04:36.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:36 smithi076 ceph-mon[35642]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd65bf47-57d6-49f1-ac41-49ef945e9bf5"}]': finished 2024-01-25T16:04:36.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:36 smithi076 ceph-mon[35642]: osdmap e78: 12 total, 11 up, 12 in 2024-01-25T16:04:36.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:36 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:37.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:37 smithi033 ceph-mon[31417]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:04:37.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:37 smithi033 ceph-mon[31417]: from='client.? 172.21.15.152:0/2500317491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:37 smithi152 ceph-mon[36015]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:04:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:37 smithi152 ceph-mon[36015]: from='client.? 172.21.15.152:0/2500317491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:37.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:37 smithi076 ceph-mon[35642]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-01-25T16:04:37.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:37 smithi076 ceph-mon[35642]: from='client.? 172.21.15.152:0/2500317491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-01-25T16:04:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:39 smithi076 ceph-mon[35642]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:04:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:39 smithi033 ceph-mon[31417]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:04:39.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:39 smithi152 ceph-mon[36015]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-01-25T16:04:41.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:41 smithi033 ceph-mon[31417]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:04:41.371 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:41 smithi152 ceph-mon[36015]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:04:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:41 smithi076 ceph-mon[35642]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-01-25T16:04:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:43 smithi076 ceph-mon[35642]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:04:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:43 smithi033 ceph-mon[31417]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:04:43.608 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:43 smithi152 ceph-mon[36015]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-01-25T16:04:44.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-25T16:04:44.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:44.445 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-25T16:04:44.445 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-01-25T16:04:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:45.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:45 smithi076 ceph-mon[35642]: Deploying daemon osd.11 on smithi152 2024-01-25T16:04:45.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:45 smithi076 ceph-mon[35642]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:04:45.455 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:45 smithi152 ceph-mon[36015]: Deploying daemon osd.11 on smithi152 2024-01-25T16:04:45.455 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:45 smithi152 ceph-mon[36015]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:04:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:45 smithi033 ceph-mon[31417]: Deploying daemon osd.11 on smithi152 2024-01-25T16:04:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:45 smithi033 ceph-mon[31417]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-01-25T16:04:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:47 smithi076 ceph-mon[35642]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:47.445 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:47 smithi152 ceph-mon[36015]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:47 smithi033 ceph-mon[31417]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:49 smithi076 ceph-mon[35642]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:49 smithi152 ceph-mon[36015]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.419 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:49 smithi033 ceph-mon[31417]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:51.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:51 smithi033 ceph-mon[31417]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:51.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:51 smithi152 ceph-mon[36015]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:51.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:51 smithi076 ceph-mon[35642]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:52.839 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.835+0000 7fb03b7fe700 1 -- 172.21.15.152:0/3621127020 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb04c02d540 con 0x7fb0340a1370 2024-01-25T16:04:52.839 INFO:teuthology.orchestra.run.smithi152.stdout:Created osd(s) 11 on host 'smithi152' 2024-01-25T16:04:52.840 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.837+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 msgr2=0x7fb0340a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.837+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 0x7fb0340a3830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb04c1017c0 tx=0x7fb03c00b410 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.837+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 msgr2=0x7fb04c19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.837+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb044004d30 tx=0x7fb044004d60 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 shutdown_connections 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0340a1370 0x7fb0340a3830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb04c102300 0x7fb04c19dc00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb04c100760 0x7fb04c196640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 --2- 172.21.15.152:0/3621127020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb04c101960 0x7fb04c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 >> 172.21.15.152:0/3621127020 conn(0x7fb04c0fbce0 msgr2=0x7fb04c104b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 shutdown_connections 2024-01-25T16:04:52.841 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:04:52.838+0000 7fb050aa2700 1 -- 172.21.15.152:0/3621127020 wait complete. 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:53 smithi152 ceph-mon[36015]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:53 smithi076 ceph-mon[35642]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.485 DEBUG:teuthology.orchestra.run.smithi152:osd.11> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.11.service 2024-01-25T16:04:53.488 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-01-25T16:04:53.488 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd stat -f json 2024-01-25T16:04:53.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:53.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.519 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:53 smithi033 ceph-mon[31417]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-01-25T16:04:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: osdmap e79: 12 total, 11 up, 12 in 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.031 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-mon[36015]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:55.032 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:04:54.829+0000 7f067206c700 -1 osd.11 0 waiting for initial osdmap 2024-01-25T16:04:55.032 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:04:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:04:54.845+0000 7f066e667700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-25T16:04:55.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-25T16:04:55.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: osdmap e79: 12 total, 11 up, 12 in 2024-01-25T16:04:55.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:55.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:54 smithi076 ceph-mon[35642]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: osdmap e79: 12 total, 11 up, 12 in 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: from='osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]: dispatch 2024-01-25T16:04:55.181 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:54 smithi033 ceph-mon[31417]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 -- 172.21.15.33:0/2125394993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58101830 msgr2=0x7f4d58101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2125394993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58101830 0x7f4d58101c50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4d480099c0 tx=0x7f4d48009cd0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 -- 172.21.15.33:0/2125394993 shutdown_connections 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2125394993 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d581033d0 0x7f4d58109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2125394993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58101830 0x7f4d58101c50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2125394993 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d58102a30 0x7f4d58102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 -- 172.21.15.33:0/2125394993 >> 172.21.15.33:0/2125394993 conn(0x7f4d580fcd90 msgr2=0x7f4d580ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 -- 172.21.15.33:0/2125394993 shutdown_connections 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.389+0000 7f4d5f789700 1 -- 172.21.15.33:0/2125394993 wait complete. 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 Processor -- start 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 -- start start 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101830 0x7f4d58199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d581033d0 0x7f4d581a0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a1520 con 0x7f4d58102a30 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a1660 con 0x7f4d581033d0 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5f789700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d581a17f0 con 0x7f4d58101830 2024-01-25T16:04:55.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37768/0 (socket says 172.21.15.33:37768) 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5d525700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101830 0x7f4d58199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 -- 172.21.15.33:0/2455423277 learned_addr learned my addr 172.21.15.33:0/2455423277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5dd26700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d581033d0 0x7f4d581a0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 -- 172.21.15.33:0/2455423277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101830 msgr2=0x7f4d58199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101830 0x7f4d58199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 -- 172.21.15.33:0/2455423277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d581033d0 msgr2=0x7f4d581a0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d581033d0 0x7f4d581a0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 -- 172.21.15.33:0/2455423277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480095f0 con 0x7f4d58102a30 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.390+0000 7f4d5cd24700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9d0 tx=0x7f4d5400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.391+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54009790 con 0x7f4d58102a30 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.391+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d5400ee70 con 0x7f4d58102a30 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.391+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54010ea0 con 0x7f4d58102a30 2024-01-25T16:04:55.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.391+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d581a1ad0 con 0x7f4d58102a30 2024-01-25T16:04:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.391+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d581a1f80 con 0x7f4d58102a30 2024-01-25T16:04:55.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.392+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d58068510 con 0x7f4d58102a30 2024-01-25T16:04:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.393+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4d5400cbd0 con 0x7f4d58102a30 2024-01-25T16:04:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.393+0000 7f4d4e7fc700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 0x7f4d440a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.393+0000 7f4d5d525700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 0x7f4d440a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.393+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7f4d54014070 con 0x7f4d58102a30 2024-01-25T16:04:55.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.394+0000 7f4d5d525700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 0x7f4d440a38b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009990 tx=0x7f4d48000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:55.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.397+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4d540963c0 con 0x7f4d58102a30 2024-01-25T16:04:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.598+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f4d5810d3a0 con 0x7f4d58102a30 2024-01-25T16:04:55.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.598+0000 7f4d4e7fc700 1 -- 172.21.15.33:0/2455423277 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4d540260a0 con 0x7f4d58102a30 2024-01-25T16:04:55.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 msgr2=0x7f4d440a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 0x7f4d440a38b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009990 tx=0x7f4d48000bc0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 msgr2=0x7f4d58199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400e9d0 tx=0x7f4d5400ece0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 shutdown_connections 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4d440a13f0 0x7f4d440a38b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d581033d0 0x7f4d581a0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4d58101830 0x7f4d58199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 --2- 172.21.15.33:0/2455423277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d58102a30 0x7f4d58199ed0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 >> 172.21.15.33:0/2455423277 conn(0x7f4d580fcd90 msgr2=0x7f4d58073a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 shutdown_connections 2024-01-25T16:04:55.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:55.600+0000 7f4d5f789700 1 -- 172.21.15.33:0/2455423277 wait complete. 2024-01-25T16:04:56.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:55 smithi152 ceph-mon[36015]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:04:56.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:55 smithi152 ceph-mon[36015]: osdmap e80: 12 total, 11 up, 12 in 2024-01-25T16:04:56.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:55 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:55 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2455423277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:04:56.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:55 smithi076 ceph-mon[35642]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:04:56.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:55 smithi076 ceph-mon[35642]: osdmap e80: 12 total, 11 up, 12 in 2024-01-25T16:04:56.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:55 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:55 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2455423277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:04:56.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:55 smithi033 ceph-mon[31417]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi152", "root=default"]}]': finished 2024-01-25T16:04:56.209 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:55 smithi033 ceph-mon[31417]: osdmap e80: 12 total, 11 up, 12 in 2024-01-25T16:04:56.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:55 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.210 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:55 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2455423277' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:04:56.236 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1706198670,"num_in_osds":12,"osd_in_since":1706198675,"num_remapped_pgs":0} 2024-01-25T16:04:56.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: purged_snaps scrub starts 2024-01-25T16:04:56.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: purged_snaps scrub ok 2024-01-25T16:04:56.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] boot 2024-01-25T16:04:56.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: osdmap e81: 12 total, 12 up, 12 in 2024-01-25T16:04:56.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:56.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:56 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: purged_snaps scrub starts 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: purged_snaps scrub ok 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] boot 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: osdmap e81: 12 total, 12 up, 12 in 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:57.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:57.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:56 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd stat -f json 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: purged_snaps scrub starts 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: purged_snaps scrub ok 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: osd.11 [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] boot 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: osdmap e81: 12 total, 12 up, 12 in 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:04:57.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:04:57.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:04:57.264 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:56 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:04:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:04:57.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:57 smithi033 ceph-mon[31417]: Detected new or changed devices on smithi152 2024-01-25T16:04:57.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:57 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 4270M 2024-01-25T16:04:57.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:57 smithi033 ceph-mon[31417]: osdmap e82: 12 total, 12 up, 12 in 2024-01-25T16:04:58.132 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:57 smithi152 ceph-mon[36015]: Detected new or changed devices on smithi152 2024-01-25T16:04:58.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:57 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 4270M 2024-01-25T16:04:58.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:57 smithi152 ceph-mon[36015]: osdmap e82: 12 total, 12 up, 12 in 2024-01-25T16:04:58.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:57 smithi076 ceph-mon[35642]: Detected new or changed devices on smithi152 2024-01-25T16:04:58.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:57 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 4270M 2024-01-25T16:04:58.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:57 smithi076 ceph-mon[35642]: osdmap e82: 12 total, 12 up, 12 in 2024-01-25T16:04:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/3597594060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01033e0 msgr2=0x7f8ef0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/3597594060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01033e0 0x7f8ef0109c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80099c0 tx=0x7f8ee8009cd0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/3597594060 shutdown_connections 2024-01-25T16:04:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/3597594060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef01033e0 0x7f8ef0109c90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/3597594060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef0101840 0x7f8ef0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/3597594060 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0102a40 0x7f8ef0102ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/3597594060 >> 172.21.15.33:0/3597594060 conn(0x7f8ef00fcde0 msgr2=0x7f8ef00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:59.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/3597594060 shutdown_connections 2024-01-25T16:04:59.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.090+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/3597594060 wait complete. 2024-01-25T16:04:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 Processor -- start 2024-01-25T16:04:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 -- start start 2024-01-25T16:04:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101840 0x7f8ef0197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:59.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef01033e0 0x7f8ef019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019f1c0 con 0x7f8ef0102a40 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019f300 con 0x7f8ef01033e0 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8ef4d7b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef019f490 con 0x7f8ef0101840 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8eedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8eedd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39900/0 (socket says 172.21.15.33:39900) 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8eedd9b700 1 -- 172.21.15.33:0/2694583432 learned_addr learned my addr 172.21.15.33:0/2694583432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8eee59c700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101840 0x7f8ef0197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.091+0000 7f8eeed9d700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef01033e0 0x7f8ef019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 -- 172.21.15.33:0/2694583432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101840 msgr2=0x7f8ef0197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101840 0x7f8ef0197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 -- 172.21.15.33:0/2694583432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef01033e0 msgr2=0x7f8ef019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef01033e0 0x7f8ef019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 -- 172.21.15.33:0/2694583432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee80095f0 con 0x7f8ef0102a40 2024-01-25T16:04:59.098 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:58 smithi033 ceph-mon[31417]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:04:59.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8eedd9b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d980 tx=0x7f8ed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80098d0 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ed8010460 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef019f770 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800f620 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.092+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef019fc70 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.093+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef0108ba0 con 0x7f8ef0102a40 2024-01-25T16:04:59.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.095+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8ed8010860 con 0x7f8ef0102a40 2024-01-25T16:04:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.096+0000 7f8ee77fe700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 0x7f8edc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:04:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.096+0000 7f8eee59c700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 0x7f8edc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:04:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.096+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ed80cc420 con 0x7f8ef0102a40 2024-01-25T16:04:59.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.096+0000 7f8eee59c700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 0x7f8edc0a3830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009550 tx=0x7f8ee00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:04:59.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.100+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8ed8095d40 con 0x7f8ef0102a40 2024-01-25T16:04:59.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:58 smithi152 ceph-mon[36015]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:04:59.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:58 smithi076 ceph-mon[35642]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:04:59.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.302+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f8ef00684a0 con 0x7f8ef0102a40 2024-01-25T16:04:59.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.303+0000 7f8ee77fe700 1 -- 172.21.15.33:0/2694583432 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) v1 ==== 74+0+133 (secure 0 0 0) 0x7f8ed8099360 con 0x7f8ef0102a40 2024-01-25T16:04:59.307 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 msgr2=0x7f8edc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 0x7f8edc0a3830 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0009550 tx=0x7f8ee00091b0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 msgr2=0x7f8ef0197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ed800d980 tx=0x7f8ed800dc90 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 shutdown_connections 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8edc0a1370 0x7f8edc0a3830 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef01033e0 0x7f8ef019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8ef0101840 0x7f8ef0197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 --2- 172.21.15.33:0/2694583432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8ef0102a40 0x7f8ef0197b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 >> 172.21.15.33:0/2694583432 conn(0x7f8ef00fcde0 msgr2=0x7f8ef0105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 shutdown_connections 2024-01-25T16:04:59.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:04:59.305+0000 7f8ef4d7b700 1 -- 172.21.15.33:0/2694583432 wait complete. 2024-01-25T16:04:59.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:04:59 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2694583432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:04:59.993 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1706198695,"num_in_osds":12,"osd_in_since":1706198675,"num_remapped_pgs":0} 2024-01-25T16:04:59.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd dump --format=json 2024-01-25T16:05:00.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:04:59 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2694583432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:05:00.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:04:59 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2694583432' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-01-25T16:05:00.274 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:05:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:01 smithi033 ceph-mon[31417]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:01.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:01 smithi152 ceph-mon[36015]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:01 smithi076 ceph-mon[35642]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 -- 172.21.15.33:0/763482657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074101bf0 msgr2=0x7fe07410d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 --2- 172.21.15.33:0/763482657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074101bf0 0x7fe07410d5c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe06400b0a0 tx=0x7fe06400b3b0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 -- 172.21.15.33:0/763482657 shutdown_connections 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 --2- 172.21.15.33:0/763482657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074101bf0 0x7fe07410d5c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 --2- 172.21.15.33:0/763482657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074105720 0x7fe074105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 --2- 172.21.15.33:0/763482657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 0x7fe0741016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 -- 172.21.15.33:0/763482657 >> 172.21.15.33:0/763482657 conn(0x7fe074075a40 msgr2=0x7fe074075e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 -- 172.21.15.33:0/763482657 shutdown_connections 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.907+0000 7fe079354700 1 -- 172.21.15.33:0/763482657 wait complete. 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 Processor -- start 2024-01-25T16:05:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 -- start start 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 0x7fe07419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074101bf0 0x7fe07419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe074198310 con 0x7fe074105720 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe074198480 con 0x7fe074101230 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe079354700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe074198640 con 0x7fe074101bf0 2024-01-25T16:05:01.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe0737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe0737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39910/0 (socket says 172.21.15.33:39910) 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe0737fe700 1 -- 172.21.15.33:0/2598317458 learned_addr learned my addr 172.21.15.33:0/2598317458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.908+0000 7fe072ffd700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 0x7fe07419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0727fc700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074101bf0 0x7fe07419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 -- 172.21.15.33:0/2598317458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074101bf0 msgr2=0x7fe07419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074101bf0 0x7fe07419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 -- 172.21.15.33:0/2598317458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 msgr2=0x7fe07419da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 0x7fe07419da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 -- 172.21.15.33:0/2598317458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe064009d00 con 0x7fe074105720 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe0737fe700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe06400b070 tx=0x7fe064007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06400e040 con 0x7fe074105720 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0741988c0 con 0x7fe074105720 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe064009440 con 0x7fe074105720 2024-01-25T16:05:01.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06401b450 con 0x7fe074105720 2024-01-25T16:05:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.909+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0741a49c0 con 0x7fe074105720 2024-01-25T16:05:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.911+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe074109d20 con 0x7fe074105720 2024-01-25T16:05:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.911+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe06401d480 con 0x7fe074105720 2024-01-25T16:05:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.912+0000 7fe06bfff700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 0x7fe0600a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.912+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe0640cdf30 con 0x7fe074105720 2024-01-25T16:05:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.912+0000 7fe072ffd700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 0x7fe0600a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:01.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.913+0000 7fe072ffd700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 0x7fe0600a3830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00abd0 tx=0x7fe06c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:01.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:01.917+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe0640978e0 con 0x7fe074105720 2024-01-25T16:05:02.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.122+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe074198fd0 con 0x7fe074105720 2024-01-25T16:05:02.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.123+0000 7fe06bfff700 1 -- 172.21.15.33:0/2598317458 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18327 (secure 0 0 0) 0x7fe06409af00 con 0x7fe074105720 2024-01-25T16:05:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:04:56.833580+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 msgr2=0x7fe0600a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 0x7fe0600a3830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00abd0 tx=0x7fe06c00a380 comp rx=0 tx=0).stop 2024-01-25T16:05:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 msgr2=0x7fe074197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe06400b070 tx=0x7fe064007870 comp rx=0 tx=0).stop 2024-01-25T16:05:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 shutdown_connections 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe0600a1370 0x7fe0600a3830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe074105720 0x7fe074197b60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe074101230 0x7fe07419da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 --2- 172.21.15.33:0/2598317458 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe074101bf0 0x7fe07419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.125+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 >> 172.21.15.33:0/2598317458 conn(0x7fe074075a40 msgr2=0x7fe074100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.126+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 shutdown_connections 2024-01-25T16:05:02.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:02.126+0000 7fe079354700 1 -- 172.21.15.33:0/2598317458 wait complete. 2024-01-25T16:05:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:02 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2598317458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:02 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2598317458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:02 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2598317458' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:02.809 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:05:02.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-01-25T16:05:03.050 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:05:03.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:03 smithi033 ceph-mon[31417]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:03 smithi152 ceph-mon[36015]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:03 smithi076 ceph-mon[35642]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 -- 172.21.15.33:0/1465510723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 msgr2=0x7fac1c10e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 --2- 172.21.15.33:0/1465510723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c10e6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fac1400b0d0 tx=0x7fac1400b3e0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 -- 172.21.15.33:0/1465510723 shutdown_connections 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 --2- 172.21.15.33:0/1465510723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c10e6a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 --2- 172.21.15.33:0/1465510723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c1077d0 0x7fac1c107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 --2- 172.21.15.33:0/1465510723 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c069ee0 0x7fac1c06a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.560+0000 7fac20b83700 1 -- 172.21.15.33:0/1465510723 >> 172.21.15.33:0/1465510723 conn(0x7fac1c076700 msgr2=0x7fac1c076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 -- 172.21.15.33:0/1465510723 shutdown_connections 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 -- 172.21.15.33:0/1465510723 wait complete. 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 Processor -- start 2024-01-25T16:05:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 -- start start 2024-01-25T16:05:04.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c069ee0 0x7fac1c1007e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:04.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:04.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 0x7fac1c101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:04.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1a6190 con 0x7fac1c06a880 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.561+0000 7fac20b83700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1a6300 con 0x7fac1c069ee0 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac20b83700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1a64c0 con 0x7fac1c1077d0 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39924/0 (socket says 172.21.15.33:39924) 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 -- 172.21.15.33:0/3792187958 learned_addr learned my addr 172.21.15.33:0/3792187958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac1ad9d700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 0x7fac1c101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac1a59c700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c069ee0 0x7fac1c1007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 -- 172.21.15.33:0/3792187958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 msgr2=0x7fac1c101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 0x7fac1c101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 -- 172.21.15.33:0/3792187958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c069ee0 msgr2=0x7fac1c1007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c069ee0 0x7fac1c1007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 -- 172.21.15.33:0/3792187958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009d00 con 0x7fac1c06a880 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac1ad9d700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 0x7fac1c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:04.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac19d9b700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d950 tx=0x7fac1000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10009b40 con 0x7fac1c06a880 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c1a67a0 con 0x7fac1c06a880 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac10009ca0 con 0x7fac1c06a880 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000f620 con 0x7fac1c06a880 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.562+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c1a6cf0 con 0x7fac1c06a880 2024-01-25T16:05:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.563+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c109d80 con 0x7fac1c06a880 2024-01-25T16:05:04.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.564+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fac10010460 con 0x7fac1c06a880 2024-01-25T16:05:04.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.565+0000 7fac0b7fe700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 0x7fac040a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:04.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.565+0000 7fac1a59c700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 0x7fac040a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:04.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.565+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fac100ccb40 con 0x7fac1c06a880 2024-01-25T16:05:04.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.565+0000 7fac1a59c700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 0x7fac040a3830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005950 tx=0x7fac0c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:04.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.568+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fac10096460 con 0x7fac1c06a880 2024-01-25T16:05:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.765+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac1c1a6930 con 0x7fac1c06a880 2024-01-25T16:05:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.766+0000 7fac0b7fe700 1 -- 172.21.15.33:0/3792187958 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac1c1a6930 con 0x7fac1c06a880 2024-01-25T16:05:04.769 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:05:04.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.767+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 msgr2=0x7fac040a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:04.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.767+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 0x7fac040a3830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac0c005950 tx=0x7fac0c0058e0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.767+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 msgr2=0x7fac1c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.767+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fac1000d950 tx=0x7fac1000dd10 comp rx=0 tx=0).stop 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 shutdown_connections 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fac040a1370 0x7fac040a3830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac1c1077d0 0x7fac1c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c069ee0 0x7fac1c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 --2- 172.21.15.33:0/3792187958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c06a880 0x7fac1c100d20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 >> 172.21.15.33:0/3792187958 conn(0x7fac1c076700 msgr2=0x7fac1c0fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 shutdown_connections 2024-01-25T16:05:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:04.768+0000 7fac20b83700 1 -- 172.21.15.33:0/3792187958 wait complete. 2024-01-25T16:05:05.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:05 smithi033 ceph-mon[31417]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:05.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3792187958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.b on smithi076 2024-01-25T16:05:05.481 INFO:tasks.cephadm:Adding mds.e on smithi076 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.h on smithi076 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.k on smithi076 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.c on smithi152 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.f on smithi152 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.i on smithi152 2024-01-25T16:05:05.482 INFO:tasks.cephadm:Adding mds.l on smithi152 2024-01-25T16:05:05.482 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l' 2024-01-25T16:05:05.514 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:05 smithi152 ceph-mon[36015]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:05.514 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:05 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3792187958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:05 smithi076 ceph-mon[35642]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:05 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3792187958' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:05.732 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:05:07.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:07 smithi033 ceph-mon[31417]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- 172.21.15.152:0/2170095651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac100710 msgr2=0x7f8dac100b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 --2- 172.21.15.152:0/2170095651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac100710 0x7f8dac100b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8da800b0a0 tx=0x7f8da800b3b0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- 172.21.15.152:0/2170095651 shutdown_connections 2024-01-25T16:05:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 --2- 172.21.15.152:0/2170095651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac1022b0 0x7f8dac108b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.428 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 --2- 172.21.15.152:0/2170095651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac100710 0x7f8dac100b30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 --2- 172.21.15.152:0/2170095651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac101910 0x7f8dac101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- 172.21.15.152:0/2170095651 >> 172.21.15.152:0/2170095651 conn(0x7f8dac0fbcb0 msgr2=0x7f8dac0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- 172.21.15.152:0/2170095651 shutdown_connections 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- 172.21.15.152:0/2170095651 wait complete. 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 Processor -- start 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.423+0000 7f8db301e700 1 -- start start 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac100710 0x7f8dac196770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1022b0 0x7f8dac19dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac19e300 con 0x7f8dac1022b0 2024-01-25T16:05:07.429 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac19e440 con 0x7f8dac100710 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac19e5d0 con 0x7f8dac101910 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:55404/0 (socket says 172.21.15.152:55404) 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 -- 172.21.15.152:0/2124765144 learned_addr learned my addr 172.21.15.152:0/2124765144 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db0dba700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac100710 0x7f8dac196770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db15bb700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1022b0 0x7f8dac19dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 -- 172.21.15.152:0/2124765144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac100710 msgr2=0x7f8dac196770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac100710 0x7f8dac196770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 -- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1022b0 msgr2=0x7f8dac19dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1022b0 0x7f8dac19dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 -- 172.21.15.152:0/2124765144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009d00 con 0x7f8dac101910 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da3fff700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00c950 tx=0x7f8d9c00cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00cea0 con 0x7f8dac101910 2024-01-25T16:05:07.430 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac19e8b0 con 0x7f8dac101910 2024-01-25T16:05:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d9c004830 con 0x7f8dac101910 2024-01-25T16:05:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.424+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c005460 con 0x7f8dac101910 2024-01-25T16:05:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.425+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac19ee00 con 0x7f8dac101910 2024-01-25T16:05:07.431 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.425+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac107bc0 con 0x7f8dac101910 2024-01-25T16:05:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.427+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f8d9c007ac0 con 0x7f8dac101910 2024-01-25T16:05:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.427+0000 7f8da1ffb700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 0x7f8d940a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.427+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8d9c0cc6b0 con 0x7f8dac101910 2024-01-25T16:05:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.427+0000 7f8db0dba700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 0x7f8d940a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:07.432 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.428+0000 7f8db0dba700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 0x7f8d940a3830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009fd0 tx=0x7f8da8006000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:07.434 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.431+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8d9c095fd0 con 0x7f8dac101910 2024-01-25T16:05:07.630 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.627+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dac02d5a0 con 0x7f8d940a1370 2024-01-25T16:05:07.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:07 smithi152 ceph-mon[36015]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:07.637 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.635+0000 7f8da1ffb700 1 -- 172.21.15.152:0/2124765144 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f8dac02d5a0 con 0x7f8d940a1370 2024-01-25T16:05:07.638 INFO:teuthology.orchestra.run.smithi152.stdout:Scheduled mds.all update... 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 msgr2=0x7f8d940a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 0x7f8d940a3830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009fd0 tx=0x7f8da8006000 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 msgr2=0x7f8dac196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00c950 tx=0x7f8d9c00cc60 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 shutdown_connections 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8d940a1370 0x7f8d940a3830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8dac1022b0 0x7f8dac19dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac100710 0x7f8dac196770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 --2- 172.21.15.152:0/2124765144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8dac101910 0x7f8dac196cb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:07.640 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 >> 172.21.15.152:0/2124765144 conn(0x7f8dac0fbcb0 msgr2=0x7f8dac104d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 shutdown_connections 2024-01-25T16:05:07.641 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-25T16:05:07.637+0000 7f8db301e700 1 -- 172.21.15.152:0/2124765144 wait complete. 2024-01-25T16:05:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:07 smithi076 ceph-mon[35642]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:08.283 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.a.service 2024-01-25T16:05:08.287 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.d.service 2024-01-25T16:05:08.330 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.g.service 2024-01-25T16:05:08.373 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.j.service 2024-01-25T16:05:08.416 DEBUG:teuthology.orchestra.run.smithi076:mds.b> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.b.service 2024-01-25T16:05:08.419 DEBUG:teuthology.orchestra.run.smithi076:mds.e> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.e.service 2024-01-25T16:05:08.463 DEBUG:teuthology.orchestra.run.smithi076:mds.h> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.h.service 2024-01-25T16:05:08.505 DEBUG:teuthology.orchestra.run.smithi076:mds.k> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.k.service 2024-01-25T16:05:08.549 DEBUG:teuthology.orchestra.run.smithi152:mds.c> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.c.service 2024-01-25T16:05:08.551 DEBUG:teuthology.orchestra.run.smithi152:mds.f> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.f.service 2024-01-25T16:05:08.594 DEBUG:teuthology.orchestra.run.smithi152:mds.i> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.i.service 2024-01-25T16:05:08.637 DEBUG:teuthology.orchestra.run.smithi152:mds.l> sudo journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.l.service 2024-01-25T16:05:08.680 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-01-25T16:05:08.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:05:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 -- 172.21.15.33:0/831506009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0101800 msgr2=0x7f05a0103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 --2- 172.21.15.33:0/831506009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0101800 0x7f05a0103bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0598009900 tx=0x7f059802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:08.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 -- 172.21.15.33:0/831506009 shutdown_connections 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 --2- 172.21.15.33:0/831506009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0106a60 0x7f05a0108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 --2- 172.21.15.33:0/831506009 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 0x7f05a0106520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 --2- 172.21.15.33:0/831506009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0101800 0x7f05a0103bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 -- 172.21.15.33:0/831506009 >> 172.21.15.33:0/831506009 conn(0x7f05a00fb1d0 msgr2=0x7f05a00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 -- 172.21.15.33:0/831506009 shutdown_connections 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.791+0000 7f05a4f83640 1 -- 172.21.15.33:0/831506009 wait complete. 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 Processor -- start 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 -- start start 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 0x7f05a0101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 0x7f05a00ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0100450 con 0x7f05a0106a60 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a01005c0 con 0x7f05a0101800 2024-01-25T16:05:08.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f05a4f83640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a01a0d60 con 0x7f05a0104130 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f059ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33450/0 (socket says 172.21.15.33:33450) 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f059ed9d640 1 -- 172.21.15.33:0/2800145305 learned_addr learned my addr 172.21.15.33:0/2800145305 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.792+0000 7f059dd9b640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 0x7f05a00ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059e59c640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 0x7f05a0101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 -- 172.21.15.33:0/2800145305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 msgr2=0x7f05a00ff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 0x7f05a00ff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 -- 172.21.15.33:0/2800145305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 msgr2=0x7f05a0101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 0x7f05a0101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 -- 172.21.15.33:0/2800145305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009560 con 0x7f05a0106a60 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059e59c640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 0x7f05a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:08.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f059ed9d640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f059000ead0 tx=0x7f059000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000cde0 con 0x7f05a0106a60 2024-01-25T16:05:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0590005020 con 0x7f05a0106a60 2024-01-25T16:05:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a01a0fb0 con 0x7f05a0106a60 2024-01-25T16:05:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900116d0 con 0x7f05a0106a60 2024-01-25T16:05:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.793+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a01a1440 con 0x7f05a0106a60 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.794+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a00fccc0 con 0x7f05a0106a60 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.795+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0590005190 con 0x7f05a0106a60 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.796+0000 7f05977fe640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 0x7f058c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.796+0000 7f059e59c640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 0x7f058c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.796+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05900cc940 con 0x7f05a0106a60 2024-01-25T16:05:08.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.796+0000 7f059e59c640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 0x7f058c0a37a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0598006010 tx=0x7f059802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:08.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.800+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f05900961c0 con 0x7f05a0106a60 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:08.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:08 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:08.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-01-25T16:05:08.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:08.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:08 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.979+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05a002cb70 con 0x7f05a0106a60 2024-01-25T16:05:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.980+0000 7f05977fe640 1 -- 172.21.15.33:0/2800145305 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f05900997e0 con 0x7f05a0106a60 2024-01-25T16:05:08.983 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:08.983 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:04:56.833580+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 msgr2=0x7f058c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 0x7f058c0a37a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0598006010 tx=0x7f059802c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 msgr2=0x7f05a00ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f059000ead0 tx=0x7f059000ef90 comp rx=0 tx=0).stop 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 shutdown_connections 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f058c0a12e0 0x7f058c0a37a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05a0106a60 0x7f05a00ffcf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05a0104130 0x7f05a00ff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 --2- 172.21.15.33:0/2800145305 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05a0101800 0x7f05a0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:08.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 >> 172.21.15.33:0/2800145305 conn(0x7f05a00fb1d0 msgr2=0x7f05a0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:08.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 shutdown_connections 2024-01-25T16:05:08.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:08.982+0000 7f05a4f83640 1 -- 172.21.15.33:0/2800145305 wait complete. 2024-01-25T16:05:08.994 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:08.995 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:08 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:08.997 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:05:08.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 -- 172.21.15.33:0/2485779070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba80ffc10 msgr2=0x7f5ba8100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 --2- 172.21.15.33:0/2485779070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba80ffc10 0x7f5ba8100030 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009900 tx=0x7f5b9802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 -- 172.21.15.33:0/2485779070 shutdown_connections 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 --2- 172.21.15.33:0/2485779070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba81017c0 0x7f5ba8108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 --2- 172.21.15.33:0/2485779070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba8100e00 0x7f5ba8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 --2- 172.21.15.33:0/2485779070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba80ffc10 0x7f5ba8100030 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 -- 172.21.15.33:0/2485779070 >> 172.21.15.33:0/2485779070 conn(0x7f5ba80fb1a0 msgr2=0x7f5ba80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 -- 172.21.15.33:0/2485779070 shutdown_connections 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.099+0000 7f5bae441640 1 -- 172.21.15.33:0/2485779070 wait complete. 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 Processor -- start 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 -- start start 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba80ffc10 0x7f5ba810c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81017c0 0x7f5ba810adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba810b520 con 0x7f5ba8100e00 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba810b690 con 0x7f5ba81017c0 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5bae441640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba81a0d70 con 0x7f5ba80ffc10 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.100+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33474/0 (socket says 172.21.15.33:33474) 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 -- 172.21.15.33:0/3396909851 learned_addr learned my addr 172.21.15.33:0/3396909851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5bac9de640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81017c0 0x7f5ba810adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba7fff640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba80ffc10 0x7f5ba810c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 -- 172.21.15.33:0/3396909851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba80ffc10 msgr2=0x7f5ba810c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba80ffc10 0x7f5ba810c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 -- 172.21.15.33:0/3396909851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81017c0 msgr2=0x7f5ba810adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81017c0 0x7f5ba810adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 -- 172.21.15.33:0/3396909851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b98009560 con 0x7f5ba8100e00 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba77fe640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000ea20 tx=0x7f5b9000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90009700 con 0x7f5ba8100e00 2024-01-25T16:05:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b9000cd80 con 0x7f5ba8100e00 2024-01-25T16:05:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba81a0fc0 con 0x7f5ba8100e00 2024-01-25T16:05:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900187e0 con 0x7f5ba8100e00 2024-01-25T16:05:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.101+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba81a14a0 con 0x7f5ba8100e00 2024-01-25T16:05:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.102+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba8067f30 con 0x7f5ba8100e00 2024-01-25T16:05:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.103+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f5b90018940 con 0x7f5ba8100e00 2024-01-25T16:05:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.104+0000 7f5ba57fa640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 0x7f5b940a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.104+0000 7f5ba7fff640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 0x7f5b940a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.104+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5b90014070 con 0x7f5ba8100e00 2024-01-25T16:05:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.104+0000 7f5ba7fff640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 0x7f5b940a37a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b9802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.108+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5b90052600 con 0x7f5ba8100e00 2024-01-25T16:05:09.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.287+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ba802cb80 con 0x7f5ba8100e00 2024-01-25T16:05:09.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.288+0000 7f5ba57fa640 1 -- 172.21.15.33:0/3396909851 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b90096320 con 0x7f5ba8100e00 2024-01-25T16:05:09.290 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:05:09.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 msgr2=0x7f5b940a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 0x7f5b940a37a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b9802c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 msgr2=0x7f5ba810a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b9000ea20 tx=0x7f5b9000eee0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 shutdown_connections 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81017c0 0x7f5ba810adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b940a12e0 0x7f5b940a37a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ba8100e00 0x7f5ba810a880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 --2- 172.21.15.33:0/3396909851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5ba80ffc10 0x7f5ba810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 >> 172.21.15.33:0/3396909851 conn(0x7f5ba80fb1a0 msgr2=0x7f5ba8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 shutdown_connections 2024-01-25T16:05:09.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.290+0000 7f5bae441640 1 -- 172.21.15.33:0/3396909851 wait complete. 2024-01-25T16:05:09.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 -- 172.21.15.33:0/3946877643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec100e00 msgr2=0x7fd8ec101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3946877643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec100e00 0x7fd8ec101280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009900 tx=0x7fd8e002bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 -- 172.21.15.33:0/3946877643 shutdown_connections 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3946877643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3946877643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec100e00 0x7fd8ec101280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3946877643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 -- 172.21.15.33:0/3946877643 >> 172.21.15.33:0/3946877643 conn(0x7fd8ec0fb180 msgr2=0x7fd8ec0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 -- 172.21.15.33:0/3946877643 shutdown_connections 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.405+0000 7fd8f2368640 1 -- 172.21.15.33:0/3946877643 wait complete. 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 Processor -- start 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 -- start start 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec100e00 0x7fd8ec1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19e7a0 con 0x7fd8ec1017c0 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19e910 con 0x7fd8ec0ffc10 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f2368640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec19ead0 con 0x7fd8ec100e00 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33486/0 (socket says 172.21.15.33:33486) 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8ebfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:38120/0 (socket says 172.21.15.33:38120) 2024-01-25T16:05:09.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 -- 172.21.15.33:0/3847710427 learned_addr learned my addr 172.21.15.33:0/3847710427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8eb7fe640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec100e00 0x7fd8ec1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 -- 172.21.15.33:0/3847710427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec100e00 msgr2=0x7fd8ec1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec100e00 0x7fd8ec1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 -- 172.21.15.33:0/3847710427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 msgr2=0x7fd8ec196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8f0905640 1 -- 172.21.15.33:0/3847710427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0009560 con 0x7fd8ec1017c0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.406+0000 7fd8ebfff640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8f0905640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400eb10 tx=0x7fd8e400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4004b20 con 0x7fd8ec1017c0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e4004c80 con 0x7fd8ec1017c0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec19edb0 con 0x7fd8ec1017c0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010690 con 0x7fd8ec1017c0 2024-01-25T16:05:09.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.407+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec19f230 con 0x7fd8ec1017c0 2024-01-25T16:05:09.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.408+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec107000 con 0x7fd8ec1017c0 2024-01-25T16:05:09.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.410+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd8e4004fd0 con 0x7fd8ec1017c0 2024-01-25T16:05:09.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.410+0000 7fd8e97fa640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 0x7fd8d40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:09.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.411+0000 7fd8ebfff640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 0x7fd8d40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:09.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.411+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd8e4014070 con 0x7fd8ec1017c0 2024-01-25T16:05:09.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.411+0000 7fd8ebfff640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 0x7fd8d40a37a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec197b60 tx=0x7fd8dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:09.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.413+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd8e40966e0 con 0x7fd8ec1017c0 2024-01-25T16:05:09.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.635+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8ec10c600 con 0x7fd8ec1017c0 2024-01-25T16:05:09.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.636+0000 7fd8e97fa640 1 -- 172.21.15.33:0/3847710427 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fd8e4099d00 con 0x7fd8ec1017c0 2024-01-25T16:05:09.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:09.639 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-25T16:05:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 msgr2=0x7fd8d40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 0x7fd8d40a37a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec197b60 tx=0x7fd8dc005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 msgr2=0x7fd8ec19e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400eb10 tx=0x7fd8e400c2d0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 shutdown_connections 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd8d40a12e0 0x7fd8d40a37a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ec1017c0 0x7fd8ec19e230 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8ec100e00 0x7fd8ec1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 --2- 172.21.15.33:0/3847710427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec0ffc10 0x7fd8ec196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 >> 172.21.15.33:0/3847710427 conn(0x7fd8ec0fb180 msgr2=0x7fd8ec1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 shutdown_connections 2024-01-25T16:05:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:09.638+0000 7fd8f2368640 1 -- 172.21.15.33:0/3847710427 wait complete. 2024-01-25T16:05:09.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:09 smithi076 ceph-mon[35642]: Deploying daemon mds.l on smithi152 2024-01-25T16:05:09.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:09 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2800145305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:09.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:09 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3396909851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:09.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:09 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3847710427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:10.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:09 smithi033 ceph-mon[31417]: Deploying daemon mds.l on smithi152 2024-01-25T16:05:10.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:09 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2800145305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:10.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:09 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3396909851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:10.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:09 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3847710427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:10.117 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:09 smithi152 ceph-mon[36015]: Deploying daemon mds.l on smithi152 2024-01-25T16:05:10.117 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:09 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2800145305' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:10.117 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:09 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3396909851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:10.117 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:09 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3847710427' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:10.822 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:10 smithi152 systemd[1]: Starting Ceph mds.l for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:11.097 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:11 smithi152 ceph-mon[36015]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:11.097 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:10 smithi152 podman[65966]: 2024-01-25 16:05:10.722045574 +0000 UTC m=+0.017818440 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:11.097 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:10 smithi152 podman[65966]: 2024-01-25 16:05:10.850910897 +0000 UTC m=+0.146683754 container create 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:05:11.097 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 podman[65966]: 2024-01-25 16:05:11.095238907 +0000 UTC m=+0.391011781 container init 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-01-25T16:05:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:11 smithi033 ceph-mon[31417]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:11.336 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:11 smithi076 ceph-mon[35642]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:11.383 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 podman[65966]: 2024-01-25 16:05:11.098542734 +0000 UTC m=+0.394315608 container start 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-01-25T16:05:11.383 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 ceph-mds[65985]: -- 172.21.15.152:0/2022851416 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c021b3b520 con 0x55c021b11000 2024-01-25T16:05:11.383 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l[65981]: starting mds.l at 2024-01-25T16:05:11.383 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 bash[65966]: 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b 2024-01-25T16:05:11.383 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:05:11 smithi152 systemd[1]: Started Ceph mds.l for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:11.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:11.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 -- 172.21.15.33:0/1010981413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc101850 msgr2=0x7f59fc103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 --2- 172.21.15.33:0/1010981413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc101850 0x7f59fc103c40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009900 tx=0x7f59ec02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 -- 172.21.15.33:0/1010981413 shutdown_connections 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 --2- 172.21.15.33:0/1010981413 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc106ab0 0x7f59fc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 --2- 172.21.15.33:0/1010981413 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc104180 0x7f59fc106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 --2- 172.21.15.33:0/1010981413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc101850 0x7f59fc103c40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 -- 172.21.15.33:0/1010981413 >> 172.21.15.33:0/1010981413 conn(0x7f59fc0fb1a0 msgr2=0x7f59fc0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 -- 172.21.15.33:0/1010981413 shutdown_connections 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.766+0000 7f5a00cec640 1 -- 172.21.15.33:0/1010981413 wait complete. 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 Processor -- start 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 -- start start 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc104180 0x7f59fc197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19e820 con 0x7f59fc106ab0 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19e990 con 0x7f59fc104180 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f5a00cec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc19eb50 con 0x7f59fc101850 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59f3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc104180 0x7f59fc197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33502/0 (socket says 172.21.15.33:33502) 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 -- 172.21.15.33:0/3751657661 learned_addr learned my addr 172.21.15.33:0/3751657661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fa59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:55178/0 (socket says 172.21.15.33:55178) 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 -- 172.21.15.33:0/3751657661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 msgr2=0x7f59fc196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 -- 172.21.15.33:0/3751657661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc104180 msgr2=0x7f59fc197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:11.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc104180 0x7f59fc197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fad9d640 1 -- 172.21.15.33:0/3751657661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec009560 con 0x7f59fc106ab0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.767+0000 7f59fa59c640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f59fad9d640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f59f400eb10 tx=0x7f59f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4004b20 con 0x7f59fc106ab0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59f4004c80 con 0x7f59fc106ab0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc19ee30 con 0x7f59fc106ab0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f4010690 con 0x7f59fc106ab0 2024-01-25T16:05:11.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.768+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc19f200 con 0x7f59fc106ab0 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.769+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc0fcc90 con 0x7f59fc106ab0 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.770+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f59f4005020 con 0x7f59fc106ab0 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.770+0000 7f59f37fe640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 0x7f59e80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.771+0000 7f59fa59c640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 0x7f59e80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.771+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f59f4014070 con 0x7f59fc106ab0 2024-01-25T16:05:11.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.771+0000 7f59fa59c640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 0x7f59e80a37a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005ab0 tx=0x7f59ec02c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.775+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f59f4096530 con 0x7f59fc106ab0 2024-01-25T16:05:11.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.990+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59fc10c610 con 0x7f59fc106ab0 2024-01-25T16:05:11.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.991+0000 7f59f37fe640 1 -- 172.21.15.33:0/3751657661 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f59f4098460 con 0x7f59fc106ab0 2024-01-25T16:05:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:11.994 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-01-25T16:05:11.994 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-01-25T16:05:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.992+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 msgr2=0x7f59e80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.992+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 0x7f59e80a37a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f59ec005ab0 tx=0x7f59ec02c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 msgr2=0x7f59fc19e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:11.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f59f400eb10 tx=0x7f59f400c2d0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 shutdown_connections 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f59e80a12e0 0x7f59e80a37a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59fc106ab0 0x7f59fc19e2b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc104180 0x7f59fc197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 --2- 172.21.15.33:0/3751657661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f59fc101850 0x7f59fc196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 >> 172.21.15.33:0/3751657661 conn(0x7f59fc0fb1a0 msgr2=0x7f59fc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 shutdown_connections 2024-01-25T16:05:11.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:11.993+0000 7f5a00cec640 1 -- 172.21.15.33:0/3751657661 wait complete. 2024-01-25T16:05:12.142 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6832/3591759228,v1:172.21.15.152:6833/3591759228] up:boot 2024-01-25T16:05:12.142 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: fsmap 1 up:standby 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:12.143 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:12 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3751657661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:12.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6832/3591759228,v1:172.21.15.152:6833/3591759228] up:boot 2024-01-25T16:05:12.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: fsmap 1 up:standby 2024-01-25T16:05:12.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:12.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3751657661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6832/3591759228,v1:172.21.15.152:6833/3591759228] up:boot 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: fsmap 1 up:standby 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:12.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:12 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3751657661' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:13.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:13 smithi076 ceph-mon[35642]: Deploying daemon mds.k on smithi076 2024-01-25T16:05:13.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:13 smithi076 ceph-mon[35642]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:13 smithi033 ceph-mon[31417]: Deploying daemon mds.k on smithi076 2024-01-25T16:05:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:13 smithi033 ceph-mon[31417]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:13 smithi152 ceph-mon[36015]: Deploying daemon mds.k on smithi076 2024-01-25T16:05:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:13 smithi152 ceph-mon[36015]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:13.666 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 systemd[1]: Starting Ceph mds.k for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:13.918 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 podman[65629]: 2024-01-25 16:05:13.565837627 +0000 UTC m=+0.017469569 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:13.918 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 podman[65629]: 2024-01-25 16:05:13.685531185 +0000 UTC m=+0.137163163 container create 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:05:14.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 -- 172.21.15.33:0/2496925897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc50100f50 msgr2=0x7fdc501077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 --2- 172.21.15.33:0/2496925897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc50100f50 0x7fdc501077e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009ca0 tx=0x7fdc4802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 -- 172.21.15.33:0/2496925897 shutdown_connections 2024-01-25T16:05:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 --2- 172.21.15.33:0/2496925897 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc50100f50 0x7fdc501077e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 --2- 172.21.15.33:0/2496925897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100590 0x7fdc50100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 --2- 172.21.15.33:0/2496925897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc500ff3a0 0x7fdc500ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 -- 172.21.15.33:0/2496925897 >> 172.21.15.33:0/2496925897 conn(0x7fdc500fa910 msgr2=0x7fdc500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 -- 172.21.15.33:0/2496925897 shutdown_connections 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.119+0000 7fdc5511d640 1 -- 172.21.15.33:0/2496925897 wait complete. 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 Processor -- start 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 -- start start 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 0x7fdc50197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100f50 0x7fdc5019e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5019e730 con 0x7fdc500ff3a0 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5019e8a0 con 0x7fdc50100590 2024-01-25T16:05:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc5511d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5019ea60 con 0x7fdc50100f50 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc4ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc4ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33512/0 (socket says 172.21.15.33:33512) 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc4ed9d640 1 -- 172.21.15.33:0/1263039944 learned_addr learned my addr 172.21.15.33:0/1263039944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.120+0000 7fdc4e59c640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 0x7fdc50197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4f59e640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100f50 0x7fdc5019e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 -- 172.21.15.33:0/1263039944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100f50 msgr2=0x7fdc5019e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100f50 0x7fdc5019e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 -- 172.21.15.33:0/1263039944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 msgr2=0x7fdc50197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 0x7fdc50197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 -- 172.21.15.33:0/1263039944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc48009900 con 0x7fdc500ff3a0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4e59c640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 0x7fdc50197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc4ed9d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d770 tx=0x7fdc4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40009c30 con 0x7fdc500ff3a0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc5019ed40 con 0x7fdc500ff3a0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc4000de80 con 0x7fdc500ff3a0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40003e20 con 0x7fdc500ff3a0 2024-01-25T16:05:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.121+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5019f1e0 con 0x7fdc500ff3a0 2024-01-25T16:05:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.122+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc50106740 con 0x7fdc500ff3a0 2024-01-25T16:05:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.123+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdc40003f80 con 0x7fdc500ff3a0 2024-01-25T16:05:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.124+0000 7fdc37fff640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 0x7fdc380a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.124+0000 7fdc4e59c640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 0x7fdc380a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.124+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdc400cc220 con 0x7fdc500ff3a0 2024-01-25T16:05:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.124+0000 7fdc4e59c640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 0x7fdc380a37a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005fd0 tx=0x7fdc4400b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.128+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdc40095ab0 con 0x7fdc500ff3a0 2024-01-25T16:05:14.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.347+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc5010c8d0 con 0x7fdc500ff3a0 2024-01-25T16:05:14.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.348+0000 7fdc37fff640 1 -- 172.21.15.33:0/1263039944 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7fdc400979e0 con 0x7fdc500ff3a0 2024-01-25T16:05:14.350 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:14.350 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-25T16:05:14.351 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-01-25T16:05:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 msgr2=0x7fdc380a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 0x7fdc380a37a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc44005fd0 tx=0x7fdc4400b5f0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 msgr2=0x7fdc50196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000d770 tx=0x7fdc4000dc30 comp rx=0 tx=0).stop 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 shutdown_connections 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc50100f50 0x7fdc5019e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc50100590 0x7fdc50197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdc380a12e0 0x7fdc380a37a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 --2- 172.21.15.33:0/1263039944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc500ff3a0 0x7fdc50196c50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 >> 172.21.15.33:0/1263039944 conn(0x7fdc500fa910 msgr2=0x7fdc50103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 shutdown_connections 2024-01-25T16:05:14.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:14.350+0000 7fdc5511d640 1 -- 172.21.15.33:0/1263039944 wait complete. 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 podman[65629]: 2024-01-25 16:05:13.946747416 +0000 UTC m=+0.398379353 container init 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 podman[65629]: 2024-01-25 16:05:13.949377019 +0000 UTC m=+0.401008942 container start 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 ceph-mds[65648]: -- 172.21.15.76:0/3848336788 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b1ba17f520 con 0x55b1ba154c00 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k[65644]: starting mds.k at 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:14 smithi076 bash[65629]: 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 2024-01-25T16:05:14.419 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:05:14 smithi076 systemd[1]: Started Ceph mds.k for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6832/3364885054,v1:172.21.15.76:6833/3364885054] up:boot 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: fsmap 2 up:standby 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.365 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:15.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:15.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:15.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: Deploying daemon mds.j on smithi033 2024-01-25T16:05:15.366 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:15 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1263039944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6832/3364885054,v1:172.21.15.76:6833/3364885054] up:boot 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: fsmap 2 up:standby 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:15.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: Deploying daemon mds.j on smithi033 2024-01-25T16:05:15.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:15 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1263039944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6832/3364885054,v1:172.21.15.76:6833/3364885054] up:boot 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: fsmap 2 up:standby 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: Deploying daemon mds.j on smithi033 2024-01-25T16:05:15.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:15 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1263039944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:16.314 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 systemd[1]: Starting Ceph mds.j for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:16.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 -- 172.21.15.33:0/2510742798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa0100e20 msgr2=0x7fafa01012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 --2- 172.21.15.33:0/2510742798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa0100e20 0x7fafa01012a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf90009900 tx=0x7faf9002bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 -- 172.21.15.33:0/2510742798 shutdown_connections 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 --2- 172.21.15.33:0/2510742798 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa01017e0 0x7fafa0108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 --2- 172.21.15.33:0/2510742798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa0100e20 0x7fafa01012a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 --2- 172.21.15.33:0/2510742798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 0x7fafa0100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 -- 172.21.15.33:0/2510742798 >> 172.21.15.33:0/2510742798 conn(0x7fafa00fb1a0 msgr2=0x7fafa00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 -- 172.21.15.33:0/2510742798 shutdown_connections 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.468+0000 7fafa852c640 1 -- 172.21.15.33:0/2510742798 wait complete. 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 Processor -- start 2024-01-25T16:05:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 -- start start 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 0x7fafa0196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa0100e20 0x7fafa01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019e7c0 con 0x7fafa01017e0 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019e930 con 0x7fafa00ffc30 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa852c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa019eaf0 con 0x7fafa0100e20 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33518/0 (socket says 172.21.15.33:33518) 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 -- 172.21.15.33:0/2691172224 learned_addr learned my addr 172.21.15.33:0/2691172224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa62c8640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 0x7fafa0196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa5ac7640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa0100e20 0x7fafa01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 -- 172.21.15.33:0/2691172224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa0100e20 msgr2=0x7fafa01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa0100e20 0x7fafa01971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 -- 172.21.15.33:0/2691172224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 msgr2=0x7fafa0196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 0x7fafa0196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 -- 172.21.15.33:0/2691172224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90009560 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7fafa6ac9640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf9800cad0 tx=0x7faf9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.469+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800ae20 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.470+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf98005020 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.470+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa019edd0 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.470+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9800f6b0 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.470+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa019f250 con 0x7fafa01017e0 2024-01-25T16:05:16.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.470+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa0107010 con 0x7fafa01017e0 2024-01-25T16:05:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.472+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faf98005190 con 0x7fafa01017e0 2024-01-25T16:05:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.472+0000 7faf977fe640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 0x7faf8c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.472+0000 7fafa62c8640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 0x7faf8c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.472+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf980cc640 con 0x7fafa01017e0 2024-01-25T16:05:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.473+0000 7fafa62c8640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 0x7faf8c0a3820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fafa0197b80 tx=0x7faf9c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.477+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faf98096010 con 0x7fafa01017e0 2024-01-25T16:05:16.674 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 podman[71905]: 2024-01-25 16:05:16.345159735 +0000 UTC m=+0.017464974 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:16.674 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 podman[71905]: 2024-01-25 16:05:16.453347214 +0000 UTC m=+0.125652446 container create 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_BRANCH=HEAD) 2024-01-25T16:05:16.674 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 podman[71905]: 2024-01-25 16:05:16.672600706 +0000 UTC m=+0.344905941 container init 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-01-25T16:05:16.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.696+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fafa002cb90 con 0x7fafa01017e0 2024-01-25T16:05:16.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.696+0000 7faf977fe640 1 -- 172.21.15.33:0/2691172224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7faf98099630 con 0x7fafa01017e0 2024-01-25T16:05:16.699 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:16.699 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-25T16:05:16.700 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 msgr2=0x7faf8c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 0x7faf8c0a3820 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fafa0197b80 tx=0x7faf9c00b390 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 msgr2=0x7fafa019e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faf9800cad0 tx=0x7faf9800cf90 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 shutdown_connections 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf8c0a1360 0x7faf8c0a3820 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa01017e0 0x7fafa019e250 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fafa0100e20 0x7fafa01971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 --2- 172.21.15.33:0/2691172224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa00ffc30 0x7fafa0196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:16.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 >> 172.21.15.33:0/2691172224 conn(0x7fafa00fb1a0 msgr2=0x7fafa01041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:16.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 shutdown_connections 2024-01-25T16:05:16.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:16.699+0000 7fafa852c640 1 -- 172.21.15.33:0/2691172224 wait complete. 2024-01-25T16:05:17.064 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 podman[71905]: 2024-01-25 16:05:16.675288155 +0000 UTC m=+0.347593386 container start 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=) 2024-01-25T16:05:17.064 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 ceph-mds[71963]: -- 172.21.15.33:0/959413771 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b986a1d520 con 0x55b9869f2800 2024-01-25T16:05:17.064 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j[71959]: starting mds.j at 2024-01-25T16:05:17.064 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 bash[71905]: 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 2024-01-25T16:05:17.065 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:05:16 smithi033 systemd[1]: Started Ceph mds.j for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2691172224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:17.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:17.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:17 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2691172224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:17.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:17 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:17.442 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2691172224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:17.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:17 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:18.365 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:18 smithi152 ceph-mon[36015]: Deploying daemon mds.d on smithi033 2024-01-25T16:05:18.366 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:18 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.33:6834/2241105736,v1:172.21.15.33:6835/2241105736] up:boot 2024-01-25T16:05:18.366 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:18 smithi152 ceph-mon[36015]: fsmap 3 up:standby 2024-01-25T16:05:18.366 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:18 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:05:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:18 smithi076 ceph-mon[35642]: Deploying daemon mds.d on smithi033 2024-01-25T16:05:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:18 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.33:6834/2241105736,v1:172.21.15.33:6835/2241105736] up:boot 2024-01-25T16:05:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:18 smithi076 ceph-mon[35642]: fsmap 3 up:standby 2024-01-25T16:05:18.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:18 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:05:18.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:18 smithi033 ceph-mon[31417]: Deploying daemon mds.d on smithi033 2024-01-25T16:05:18.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:18 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.33:6834/2241105736,v1:172.21.15.33:6835/2241105736] up:boot 2024-01-25T16:05:18.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:18 smithi033 ceph-mon[31417]: fsmap 3 up:standby 2024-01-25T16:05:18.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:18 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:05:18.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:18.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- 172.21.15.33:0/1197325603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0958100e20 msgr2=0x7f09581012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:18.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 --2- 172.21.15.33:0/1197325603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0958100e20 0x7f09581012a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0948009900 tx=0x7f094802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- 172.21.15.33:0/1197325603 shutdown_connections 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 --2- 172.21.15.33:0/1197325603 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 0x7f0958108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 --2- 172.21.15.33:0/1197325603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0958100e20 0x7f09581012a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 --2- 172.21.15.33:0/1197325603 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09580ffc30 0x7f0958100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- 172.21.15.33:0/1197325603 >> 172.21.15.33:0/1197325603 conn(0x7f09580fb1a0 msgr2=0x7f09580fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- 172.21.15.33:0/1197325603 shutdown_connections 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- 172.21.15.33:0/1197325603 wait complete. 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 Processor -- start 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.835+0000 7f09603e3640 1 -- start start 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 0x7f095819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:18.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 0x7f095819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095819e870 con 0x7f09580ffc30 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095819e9e0 con 0x7f0958100e20 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095819eba0 con 0x7f09581017e0 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59256/0 (socket says 172.21.15.33:59256) 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 -- 172.21.15.33:0/3635899445 learned_addr learned my addr 172.21.15.33:0/3635899445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e980640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 0x7f095819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095d97e640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 0x7f095819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 -- 172.21.15.33:0/3635899445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 msgr2=0x7f095819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 0x7f095819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 -- 172.21.15.33:0/3635899445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 msgr2=0x7f095819ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 0x7f095819ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 -- 172.21.15.33:0/3635899445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0948009560 con 0x7f09580ffc30 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095d97e640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 0x7f095819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:18.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f095e17f640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f095400d770 tx=0x7f095400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954009c30 con 0x7f09580ffc30 2024-01-25T16:05:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095819ee20 con 0x7f09580ffc30 2024-01-25T16:05:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095400de80 con 0x7f09580ffc30 2024-01-25T16:05:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954003e20 con 0x7f09580ffc30 2024-01-25T16:05:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.836+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095819f240 con 0x7f09580ffc30 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.838+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0958107020 con 0x7f09580ffc30 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.838+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0954003f80 con 0x7f09580ffc30 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.839+0000 7f094f7fe640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 0x7f09440a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.839+0000 7f095d97e640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 0x7f09440a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.839+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f09540cc190 con 0x7f09580ffc30 2024-01-25T16:05:18.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.840+0000 7f095d97e640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 0x7f09440a37a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0948036040 tx=0x7f0948000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:18.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:18.844+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0954095ab0 con 0x7f09580ffc30 2024-01-25T16:05:19.064 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:18 smithi033 systemd[1]: Starting Ceph mds.d for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.066+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f095810b700 con 0x7f09580ffc30 2024-01-25T16:05:19.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.067+0000 7f094f7fe640 1 -- 172.21.15.33:0/3635899445 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f09540990d0 con 0x7f09580ffc30 2024-01-25T16:05:19.069 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:19.070 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-25T16:05:19.070 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 msgr2=0x7f09440a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 0x7f09440a37a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0948036040 tx=0x7f0948000bc0 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 msgr2=0x7f0958069be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f095400d770 tx=0x7f095400dc30 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 shutdown_connections 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09581017e0 0x7f095819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0958100e20 0x7f095819ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09440a12e0 0x7f09440a37a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 --2- 172.21.15.33:0/3635899445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09580ffc30 0x7f0958069be0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 >> 172.21.15.33:0/3635899445 conn(0x7f09580fb1a0 msgr2=0x7f09581041f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 shutdown_connections 2024-01-25T16:05:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:19.069+0000 7f09603e3640 1 -- 172.21.15.33:0/3635899445 wait complete. 2024-01-25T16:05:19.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:19 smithi033 ceph-mon[31417]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:19.340 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:19 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3635899445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:19.340 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 podman[72294]: 2024-01-25 16:05:19.091614975 +0000 UTC m=+0.018875747 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:19.340 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 podman[72294]: 2024-01-25 16:05:19.207843145 +0000 UTC m=+0.135103957 container create 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:05:19.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:19 smithi152 ceph-mon[36015]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:19.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:19 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3635899445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:19 smithi076 ceph-mon[35642]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:19 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3635899445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:19.814 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 podman[72294]: 2024-01-25 16:05:19.460586704 +0000 UTC m=+0.387847475 container init 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:05:19.815 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 podman[72294]: 2024-01-25 16:05:19.463461963 +0000 UTC m=+0.390722737 container start 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-01-25T16:05:19.815 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 ceph-mds[72313]: -- 172.21.15.33:0/998035138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561dde871520 con 0x561dde846c00 2024-01-25T16:05:19.815 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d[72309]: starting mds.d at 2024-01-25T16:05:19.815 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 bash[72294]: 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c 2024-01-25T16:05:19.815 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:05:19 smithi033 systemd[1]: Started Ceph mds.d for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:20.883 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: Deploying daemon mds.e on smithi076 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.33:6836/2646618640,v1:172.21.15.33:6837/2646618640] up:boot 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: fsmap 4 up:standby 2024-01-25T16:05:20.884 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:20 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:05:20.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: Deploying daemon mds.e on smithi076 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.33:6836/2646618640,v1:172.21.15.33:6837/2646618640] up:boot 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: fsmap 4 up:standby 2024-01-25T16:05:20.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:20 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:05:21.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:21.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: Deploying daemon mds.e on smithi076 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.33:6836/2646618640,v1:172.21.15.33:6837/2646618640] up:boot 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: fsmap 4 up:standby 2024-01-25T16:05:21.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:20 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:05:21.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 -- 172.21.15.33:0/265442560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104a40 msgr2=0x7fb04810b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/265442560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104a40 0x7fb04810b360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb040009ca0 tx=0x7fb04002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 -- 172.21.15.33:0/265442560 shutdown_connections 2024-01-25T16:05:21.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/265442560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104a40 0x7fb04810b360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/265442560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104080 0x7fb048104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/265442560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 0x7fb048103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 -- 172.21.15.33:0/265442560 >> 172.21.15.33:0/265442560 conn(0x7fb0480fe4e0 msgr2=0x7fb048100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 -- 172.21.15.33:0/265442560 shutdown_connections 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.199+0000 7fb04e0ee640 1 -- 172.21.15.33:0/265442560 wait complete. 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 Processor -- start 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 -- start start 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 0x7fb048198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104a40 0x7fb0481a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481a09b0 con 0x7fb048104080 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481a0b20 con 0x7fb048102f30 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb04e0ee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481a0ce0 con 0x7fb048104a40 2024-01-25T16:05:21.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.200+0000 7fb046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59308/0 (socket says 172.21.15.33:59308) 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 -- 172.21.15.33:0/171633368 learned_addr learned my addr 172.21.15.33:0/171633368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb047fff640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104a40 0x7fb0481a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb0477fe640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 0x7fb048198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 -- 172.21.15.33:0/171633368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104a40 msgr2=0x7fb0481a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104a40 0x7fb0481a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 -- 172.21.15.33:0/171633368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 msgr2=0x7fb048198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 0x7fb048198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 -- 172.21.15.33:0/171633368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009900 con 0x7fb048104080 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb046ffd640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d950 tx=0x7fb03c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c009c30 con 0x7fb048104080 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb03c009d90 con 0x7fb048104080 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481a0fc0 con 0x7fb048104080 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c00f6b0 con 0x7fb048104080 2024-01-25T16:05:21.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.201+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a13f0 con 0x7fb048104080 2024-01-25T16:05:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.202+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04810a240 con 0x7fb048104080 2024-01-25T16:05:21.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.203+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb03c0109b0 con 0x7fb048104080 2024-01-25T16:05:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.204+0000 7fb044ff9640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 0x7fb0300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.204+0000 7fb0477fe640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 0x7fb0300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.204+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb03c0cc4b0 con 0x7fb048104080 2024-01-25T16:05:21.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.204+0000 7fb0477fe640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 0x7fb0300a37a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb038005fd0 tx=0x7fb038005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:21.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.208+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb03c095d30 con 0x7fb048104080 2024-01-25T16:05:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.426+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb04802cb70 con 0x7fb048104080 2024-01-25T16:05:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.426+0000 7fb044ff9640 1 -- 172.21.15.33:0/171633368 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7fb03c021e90 con 0x7fb048104080 2024-01-25T16:05:21.429 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:21.429 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-25T16:05:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 msgr2=0x7fb0300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 0x7fb0300a37a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb038005fd0 tx=0x7fb038005850 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 msgr2=0x7fb048199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00d950 tx=0x7fb03c00de10 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 shutdown_connections 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb048104a40 0x7fb0481a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb0300a12e0 0x7fb0300a37a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb048104080 0x7fb048199400 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 --2- 172.21.15.33:0/171633368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb048102f30 0x7fb048198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 >> 172.21.15.33:0/171633368 conn(0x7fb0480fe4e0 msgr2=0x7fb048107410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 shutdown_connections 2024-01-25T16:05:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:21.428+0000 7fb04e0ee640 1 -- 172.21.15.33:0/171633368 wait complete. 2024-01-25T16:05:21.896 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:21 smithi076 systemd[1]: Starting Ceph mds.e for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:22.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:22 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/171633368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:22.168 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:21 smithi076 podman[65946]: 2024-01-25 16:05:21.893481795 +0000 UTC m=+0.098325052 container create fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True) 2024-01-25T16:05:22.169 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:21 smithi076 podman[65946]: 2024-01-25 16:05:21.812483225 +0000 UTC m=+0.017326478 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:22.169 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 podman[65946]: 2024-01-25 16:05:22.096265871 +0000 UTC m=+0.301109122 container init fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True) 2024-01-25T16:05:22.169 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 podman[65946]: 2024-01-25 16:05:22.099029354 +0000 UTC m=+0.303872589 container start fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-25T16:05:22.169 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 ceph-mds[65965]: -- 172.21.15.76:0/9143877 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55efdf24b520 con 0x55efdf220800 2024-01-25T16:05:22.169 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: starting mds.e at 2024-01-25T16:05:22.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:22 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/171633368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:22.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:22 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/171633368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:22.668 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 bash[65946]: fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 2024-01-25T16:05:22.668 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:05:22 smithi076 systemd[1]: Started Ceph mds.e for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:23.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 -- 172.21.15.33:0/1282677675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 msgr2=0x7fdd801080d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1282677675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd801080d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009ca0 tx=0x7fdd7002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 -- 172.21.15.33:0/1282677675 shutdown_connections 2024-01-25T16:05:23.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1282677675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd801080d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1282677675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 0x7fdd80101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1282677675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 0x7fdd801000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 -- 172.21.15.33:0/1282677675 >> 172.21.15.33:0/1282677675 conn(0x7fdd800fb220 msgr2=0x7fdd800fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 -- 172.21.15.33:0/1282677675 shutdown_connections 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.555+0000 7fdd87c31640 1 -- 172.21.15.33:0/1282677675 wait complete. 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 Processor -- start 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 -- start start 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 0x7fdd80069eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 0x7fdd8019b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8019e760 con 0x7fdd80101840 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8019e8d0 con 0x7fdd800ffc90 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd87c31640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8019ea90 con 0x7fdd80100e80 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd861ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd861ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59318/0 (socket says 172.21.15.33:59318) 2024-01-25T16:05:23.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd861ce640 1 -- 172.21.15.33:0/1291741187 learned_addr learned my addr 172.21.15.33:0/1291741187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd859cd640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 0x7fdd80069eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.556+0000 7fdd851cc640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 0x7fdd8019b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 -- 172.21.15.33:0/1291741187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 msgr2=0x7fdd8019b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 0x7fdd8019b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 -- 172.21.15.33:0/1291741187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 msgr2=0x7fdd80069eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 0x7fdd80069eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 -- 172.21.15.33:0/1291741187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70009900 con 0x7fdd80101840 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd861ce640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd70037040 tx=0x7fdd700093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70034c70 con 0x7fdd80101840 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8019ed10 con 0x7fdd80101840 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd70003e30 con 0x7fdd80101840 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70004800 con 0x7fdd80101840 2024-01-25T16:05:23.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.557+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8019f210 con 0x7fdd80101840 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.558+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd80107030 con 0x7fdd80101840 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.559+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdd7003b040 con 0x7fdd80101840 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.560+0000 7fdd7effd640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 0x7fdd680a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.560+0000 7fdd859cd640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 0x7fdd680a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.560+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdd700ef0e0 con 0x7fdd80101840 2024-01-25T16:05:23.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.560+0000 7fdd859cd640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 0x7fdd680a37a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd80100ce0 tx=0x7fdd78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6834/3469230010,v1:172.21.15.76:6835/3469230010] up:boot 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: fsmap 5 up:standby 2024-01-25T16:05:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:23 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:05:23.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.563+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdd700b8a00 con 0x7fdd80101840 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6834/3469230010,v1:172.21.15.76:6835/3469230010] up:boot 2024-01-25T16:05:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: fsmap 5 up:standby 2024-01-25T16:05:23.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:23 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:05:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6834/3469230010,v1:172.21.15.76:6835/3469230010] up:boot 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: fsmap 5 up:standby 2024-01-25T16:05:23.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:23 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:05:23.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.778+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd8010b4e0 con 0x7fdd80101840 2024-01-25T16:05:23.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.779+0000 7fdd7effd640 1 -- 172.21.15.33:0/1291741187 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4492 (secure 0 0 0) 0x7fdd700bc020 con 0x7fdd80101840 2024-01-25T16:05:23.782 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:23.782 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-01-25T16:05:23.782 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-01-25T16:05:23.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.780+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 msgr2=0x7fdd680a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:23.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.780+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 0x7fdd680a37a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd80100ce0 tx=0x7fdd78005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:23.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.780+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 msgr2=0x7fdd8019dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:23.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.780+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fdd70037040 tx=0x7fdd700093d0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 shutdown_connections 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdd680a12e0 0x7fdd680a37a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd80101840 0x7fdd8019dfd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd80100e80 0x7fdd8019b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 --2- 172.21.15.33:0/1291741187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd800ffc90 0x7fdd80069eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 >> 172.21.15.33:0/1291741187 conn(0x7fdd800fb220 msgr2=0x7fdd80104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 shutdown_connections 2024-01-25T16:05:23.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:23.781+0000 7fdd87c31640 1 -- 172.21.15.33:0/1291741187 wait complete. 2024-01-25T16:05:24.563 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:24 smithi152 ceph-mon[36015]: Deploying daemon mds.f on smithi152 2024-01-25T16:05:24.563 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:24 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1291741187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:24.563 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 systemd[1]: Starting Ceph mds.f for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:24 smithi033 ceph-mon[31417]: Deploying daemon mds.f on smithi152 2024-01-25T16:05:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:24 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1291741187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:24.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:24 smithi076 ceph-mon[35642]: Deploying daemon mds.f on smithi152 2024-01-25T16:05:24.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:24 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1291741187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:24.827 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 podman[66279]: 2024-01-25 16:05:24.464284266 +0000 UTC m=+0.017690036 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:24.827 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 podman[66279]: 2024-01-25 16:05:24.588096065 +0000 UTC m=+0.141501841 container create 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-01-25T16:05:24.827 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 podman[66279]: 2024-01-25 16:05:24.798892233 +0000 UTC m=+0.352297989 container init 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:05:24.827 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 podman[66279]: 2024-01-25 16:05:24.801767891 +0000 UTC m=+0.355173647 container start 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-25T16:05:25.133 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 ceph-mds[66298]: -- 172.21.15.152:0/2921185832 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562c7b29f520 con 0x562c7b275000 2024-01-25T16:05:25.133 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: starting mds.f at 2024-01-25T16:05:25.133 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 bash[66279]: 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c 2024-01-25T16:05:25.133 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:05:24 smithi152 systemd[1]: Started Ceph mds.f for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:25.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:25.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:25 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:25.588 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:25 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:25.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:25.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:25.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:25 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:25.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 -- 172.21.15.33:0/3179776237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 msgr2=0x7fa49c103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/3179776237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c103ba0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009900 tx=0x7fa48c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 -- 172.21.15.33:0/3179776237 shutdown_connections 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/3179776237 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/3179776237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 0x7fa49c1064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/3179776237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c103ba0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 -- 172.21.15.33:0/3179776237 >> 172.21.15.33:0/3179776237 conn(0x7fa49c0fb1a0 msgr2=0x7fa49c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 -- 172.21.15.33:0/3179776237 shutdown_connections 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.907+0000 7fa4a19ec640 1 -- 172.21.15.33:0/3179776237 wait complete. 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 Processor -- start 2024-01-25T16:05:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 -- start start 2024-01-25T16:05:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 0x7fa49c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19e890 con 0x7fa49c1017b0 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19ea00 con 0x7fa49c1040e0 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4a19ec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c19ebc0 con 0x7fa49c106a10 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa49affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa49affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59332/0 (socket says 172.21.15.33:59332) 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa49affd640 1 -- 172.21.15.33:0/1192464803 learned_addr learned my addr 172.21.15.33:0/1192464803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.908+0000 7fa4927fc640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 0x7fa49c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49b7fe640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 -- 172.21.15.33:0/1192464803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 msgr2=0x7fa49c19e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c19e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 -- 172.21.15.33:0/1192464803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 msgr2=0x7fa49c1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 0x7fa49c1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 -- 172.21.15.33:0/1192464803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa48c009560 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49b7fe640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa49affd640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009230 tx=0x7fa48c004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c0043d0 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c19ee40 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa48c0054c0 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c03e450 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.909+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c19f2e0 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.910+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c0fcc90 con 0x7fa49c1017b0 2024-01-25T16:05:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.911+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa48c0035f0 con 0x7fa49c1017b0 2024-01-25T16:05:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.912+0000 7fa4a09ea640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 0x7fa4880a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.912+0000 7fa4927fc640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 0x7fa4880a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.912+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa48c0ef1c0 con 0x7fa49c1017b0 2024-01-25T16:05:25.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.912+0000 7fa4927fc640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 0x7fa4880a37a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa49c069e80 tx=0x7fa48400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:25.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:25.916+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa48c0b8ad0 con 0x7fa49c1017b0 2024-01-25T16:05:26.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.134+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa49c10c660 con 0x7fa49c1017b0 2024-01-25T16:05:26.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.134+0000 7fa4a09ea640 1 -- 172.21.15.33:0/1192464803 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5286 (secure 0 0 0) 0x7fa48c0baa00 con 0x7fa49c1017b0 2024-01-25T16:05:26.137 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:26.137 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-01-25T16:05:26.137 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 msgr2=0x7fa4880a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 0x7fa4880a37a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa49c069e80 tx=0x7fa48400b390 comp rx=0 tx=0).stop 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 msgr2=0x7fa49c196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009230 tx=0x7fa48c004ee0 comp rx=0 tx=0).stop 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 shutdown_connections 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa49c106a10 0x7fa49c19e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:26.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa49c1040e0 0x7fa49c1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:26.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa4880a12e0 0x7fa4880a37a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:26.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 --2- 172.21.15.33:0/1192464803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa49c1017b0 0x7fa49c196db0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:26.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 >> 172.21.15.33:0/1192464803 conn(0x7fa49c0fb1a0 msgr2=0x7fa49c102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:26.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 shutdown_connections 2024-01-25T16:05:26.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:26.136+0000 7fa4a19ec640 1 -- 172.21.15.33:0/1192464803 wait complete. 2024-01-25T16:05:26.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: Deploying daemon mds.c on smithi152 2024-01-25T16:05:26.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6834/1042830191,v1:172.21.15.152:6835/1042830191] up:boot 2024-01-25T16:05:26.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: fsmap 6 up:standby 2024-01-25T16:05:26.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:05:26.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:26.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:26 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1192464803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:26.622 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: Deploying daemon mds.c on smithi152 2024-01-25T16:05:26.622 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6834/1042830191,v1:172.21.15.152:6835/1042830191] up:boot 2024-01-25T16:05:26.623 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: fsmap 6 up:standby 2024-01-25T16:05:26.623 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:05:26.623 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:26.623 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:26 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1192464803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:26.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: Deploying daemon mds.c on smithi152 2024-01-25T16:05:26.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6834/1042830191,v1:172.21.15.152:6835/1042830191] up:boot 2024-01-25T16:05:26.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: fsmap 6 up:standby 2024-01-25T16:05:26.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:05:26.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:26.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:26 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1192464803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:27.383 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 systemd[1]: Starting Ceph mds.c for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:27.665 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 podman[66583]: 2024-01-25 16:05:27.392611704 +0000 UTC m=+0.113910479 container create 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:05:27.665 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 podman[66583]: 2024-01-25 16:05:27.296202826 +0000 UTC m=+0.017501572 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:27.665 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 podman[66583]: 2024-01-25 16:05:27.636957612 +0000 UTC m=+0.358256349 container init 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-01-25T16:05:27.665 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 podman[66583]: 2024-01-25 16:05:27.639476671 +0000 UTC m=+0.360775406 container start 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:05:27.665 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/1437869367 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c6679c7520 con 0x55c66799cc00 2024-01-25T16:05:28.133 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:05:28.133 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 bash[66583]: 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c 2024-01-25T16:05:28.133 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:05:27 smithi152 systemd[1]: Started Ceph mds.c for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:28.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 -- 172.21.15.33:0/311377408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a81040c0 msgr2=0x7fb7a81064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/311377408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a81040c0 0x7fb7a81064b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009900 tx=0x7fb79c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 -- 172.21.15.33:0/311377408 shutdown_connections 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/311377408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81069f0 0x7fb7a8108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/311377408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a81040c0 0x7fb7a81064b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/311377408 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a8101790 0x7fb7a8103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 -- 172.21.15.33:0/311377408 >> 172.21.15.33:0/311377408 conn(0x7fb7a80fb180 msgr2=0x7fb7a80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 -- 172.21.15.33:0/311377408 shutdown_connections 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.263+0000 7fb7adc6e640 1 -- 172.21.15.33:0/311377408 wait complete. 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 Processor -- start 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 -- start start 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81040c0 0x7fb7a8197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a81069f0 0x7fb7a819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a819e7f0 con 0x7fb7a8101790 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a819e960 con 0x7fb7a81040c0 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7adc6e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a819eb20 con 0x7fb7a81069f0 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59338/0 (socket says 172.21.15.33:59338) 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 -- 172.21.15.33:0/4119019047 learned_addr learned my addr 172.21.15.33:0/4119019047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a6ffd640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81040c0 0x7fb7a8197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a7fff640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a81069f0 0x7fb7a819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 -- 172.21.15.33:0/4119019047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a81069f0 msgr2=0x7fb7a819e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a81069f0 0x7fb7a819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 -- 172.21.15.33:0/4119019047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81040c0 msgr2=0x7fb7a8197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81040c0 0x7fb7a8197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.264+0000 7fb7a77fe640 1 -- 172.21.15.33:0/4119019047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c009560 con 0x7fb7a8101790 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7a77fe640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d770 tx=0x7fb79800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798009c30 con 0x7fb7a8101790 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb79800de80 con 0x7fb7a8101790 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004de0 con 0x7fb7a8101790 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a819ee00 con 0x7fb7a8101790 2024-01-25T16:05:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.265+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a819f1d0 con 0x7fb7a8101790 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.266+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a80fcc70 con 0x7fb7a8101790 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.267+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb798004f40 con 0x7fb7a8101790 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.268+0000 7fb7a4ff9640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 0x7fb7900a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.268+0000 7fb7a6ffd640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 0x7fb7900a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.268+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb7980cc560 con 0x7fb7a8101790 2024-01-25T16:05:28.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.268+0000 7fb7a6ffd640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 0x7fb7900a37a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02c170 tx=0x7fb79c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:28.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.271+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb798095df0 con 0x7fb7a8101790 2024-01-25T16:05:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.490+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7a802cb90 con 0x7fb7a8101790 2024-01-25T16:05:28.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.491+0000 7fb7a4ff9640 1 -- 172.21.15.33:0/4119019047 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7fb798097d20 con 0x7fb7a8101790 2024-01-25T16:05:28.493 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:28.493 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-25T16:05:28.494 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 msgr2=0x7fb7900a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 0x7fb7900a37a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb79c02c170 tx=0x7fb79c000bc0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 msgr2=0x7fb7a8196d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb79800d770 tx=0x7fb79800dc30 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 shutdown_connections 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a81069f0 0x7fb7a819e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7a81040c0 0x7fb7a8197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb7900a12e0 0x7fb7900a37a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 --2- 172.21.15.33:0/4119019047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a8101790 0x7fb7a8196d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 >> 172.21.15.33:0/4119019047 conn(0x7fb7a80fb180 msgr2=0x7fb7a81021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 shutdown_connections 2024-01-25T16:05:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:28.493+0000 7fb7adc6e640 1 -- 172.21.15.33:0/4119019047 wait complete. 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/2179148160,v1:172.21.15.152:6837/2179148160] up:boot 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: fsmap 7 up:standby 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.752 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: Deploying daemon mds.h on smithi076 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:28.753 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:28 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4119019047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:28.814 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/2179148160,v1:172.21.15.152:6837/2179148160] up:boot 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: fsmap 7 up:standby 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: Deploying daemon mds.h on smithi076 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:28.815 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:28 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4119019047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/2179148160,v1:172.21.15.152:6837/2179148160] up:boot 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: fsmap 7 up:standby 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: Deploying daemon mds.h on smithi076 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:29.035 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:28 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4119019047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:30.169 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 podman[66255]: 2024-01-25 16:05:30.149397254 +0000 UTC m=+0.138067886 container create 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:05:30.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 podman[66255]: 2024-01-25 16:05:30.377297084 +0000 UTC m=+0.365967736 container init 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_BRANCH=HEAD) 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 podman[66255]: 2024-01-25 16:05:30.379977564 +0000 UTC m=+0.368648197 container start 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 ceph-mds[66275]: -- 172.21.15.76:0/4179482834 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d35de17520 con 0x55d35ddecc00 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: starting mds.h at 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 bash[66255]: 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe 2024-01-25T16:05:30.554 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:05:30 smithi076 systemd[1]: Started Ceph mds.h for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 -- 172.21.15.33:0/2637176002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30640fefe0 msgr2=0x7f30640ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 --2- 172.21.15.33:0/2637176002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30640fefe0 0x7f30640ff400 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3054009900 tx=0x7f305402bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 -- 172.21.15.33:0/2637176002 shutdown_connections 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 --2- 172.21.15.33:0/2637176002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3064102970 0x7f3064109200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 --2- 172.21.15.33:0/2637176002 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064101fb0 0x7f3064102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 --2- 172.21.15.33:0/2637176002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30640fefe0 0x7f30640ff400 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 -- 172.21.15.33:0/2637176002 >> 172.21.15.33:0/2637176002 conn(0x7f30640fa910 msgr2=0x7f30640fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 -- 172.21.15.33:0/2637176002 shutdown_connections 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 -- 172.21.15.33:0/2637176002 wait complete. 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.624+0000 7f3069fa4640 1 Processor -- start 2024-01-25T16:05:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 -- start start 2024-01-25T16:05:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 0x7f30641018d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064102970 0x7f30641004a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064100c30 con 0x7f3064101fb0 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3064100da0 con 0x7f30640fefe0 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3069fa4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30641a0db0 con 0x7f3064102970 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40856/0 (socket says 172.21.15.33:40856) 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 -- 172.21.15.33:0/1307722474 learned_addr learned my addr 172.21.15.33:0/1307722474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f30637fe640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 0x7f30641018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3063fff640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064102970 0x7f30641004a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 -- 172.21.15.33:0/1307722474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064102970 msgr2=0x7f30641004a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064102970 0x7f30641004a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 -- 172.21.15.33:0/1307722474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 msgr2=0x7f30641018d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 0x7f30641018d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 -- 172.21.15.33:0/1307722474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3054009560 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f30637fe640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 0x7f30641018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.625+0000 7f3062ffd640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f305800ea20 tx=0x7f305800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.626+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058009720 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.626+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f305800cd80 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.626+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30641a0fb0 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.626+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058010430 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.626+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30641a1440 con 0x7f3064101fb0 2024-01-25T16:05:30.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.627+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3064108150 con 0x7f3064101fb0 2024-01-25T16:05:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.628+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3058005020 con 0x7f3064101fb0 2024-01-25T16:05:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.629+0000 7f3060ff9640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 0x7f304c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.629+0000 7f30637fe640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 0x7f304c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.629+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3058014070 con 0x7f3064101fb0 2024-01-25T16:05:30.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.629+0000 7f30637fe640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 0x7f304c0a37a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3054000c00 tx=0x7f305402c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.633+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f30580964c0 con 0x7f3064101fb0 2024-01-25T16:05:30.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.848+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f306402cb70 con 0x7f3064101fb0 2024-01-25T16:05:30.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.849+0000 7f3060ff9640 1 -- 172.21.15.33:0/1307722474 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6080 (secure 0 0 0) 0x7f3058099ae0 con 0x7f3064101fb0 2024-01-25T16:05:30.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:30.852 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-01-25T16:05:30.852 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 msgr2=0x7f304c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 0x7f304c0a37a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3054000c00 tx=0x7f305402c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 msgr2=0x7f30640fff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f305800ea20 tx=0x7f305800ee20 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 shutdown_connections 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3064102970 0x7f30641004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f304c0a12e0 0x7f304c0a37a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3064101fb0 0x7f30640fff60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 --2- 172.21.15.33:0/1307722474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30640fefe0 0x7f30641018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 >> 172.21.15.33:0/1307722474 conn(0x7f30640fa910 msgr2=0x7f3064105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 shutdown_connections 2024-01-25T16:05:30.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:30.851+0000 7f3069fa4640 1 -- 172.21.15.33:0/1307722474 wait complete. 2024-01-25T16:05:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:31.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:31.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:31 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1307722474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:31.367 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:31.368 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:31 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1307722474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:31.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:31 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1307722474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:32.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:32 smithi033 ceph-mon[31417]: Deploying daemon mds.i on smithi152 2024-01-25T16:05:32.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:32 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:boot 2024-01-25T16:05:32.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:32 smithi033 ceph-mon[31417]: fsmap 8 up:standby 2024-01-25T16:05:32.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:32 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:05:32.380 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:32 smithi152 ceph-mon[36015]: Deploying daemon mds.i on smithi152 2024-01-25T16:05:32.380 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:32 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:boot 2024-01-25T16:05:32.380 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:32 smithi152 ceph-mon[36015]: fsmap 8 up:standby 2024-01-25T16:05:32.380 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:32 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:05:32.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:32 smithi076 ceph-mon[35642]: Deploying daemon mds.i on smithi152 2024-01-25T16:05:32.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:32 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:boot 2024-01-25T16:05:32.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:32 smithi076 ceph-mon[35642]: fsmap 8 up:standby 2024-01-25T16:05:32.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:32 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:05:32.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/3343326887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc100900 msgr2=0x7f4cdc100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/3343326887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc100900 0x7f4cdc100d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009ca0 tx=0x7f4cd802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/3343326887 shutdown_connections 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/3343326887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc100900 0x7f4cdc100d80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/3343326887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 0x7f4cdc1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/3343326887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/3343326887 >> 172.21.15.33:0/3343326887 conn(0x7f4cdc0fb220 msgr2=0x7f4cdc0fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/3343326887 shutdown_connections 2024-01-25T16:05:32.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.978+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/3343326887 wait complete. 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 Processor -- start 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 -- start start 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 0x7f4cdc1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc100900 0x7f4cdc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0970 con 0x7f4cdc0ff600 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0ae0 con 0x7f4cdc100900 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce3b2c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1a0ca0 con 0x7f4cdc0fff60 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40868/0 (socket says 172.21.15.33:40868) 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 -- 172.21.15.33:0/2836552340 learned_addr learned my addr 172.21.15.33:0/2836552340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce10c7640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 0x7f4cdc1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:32.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce20c9640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc100900 0x7f4cdc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 -- 172.21.15.33:0/2836552340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 msgr2=0x7f4cdc1993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 0x7f4cdc1993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 -- 172.21.15.33:0/2836552340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc100900 msgr2=0x7f4cdc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc100900 0x7f4cdc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.979+0000 7f4ce18c8640 1 -- 172.21.15.33:0/2836552340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd8009900 con 0x7f4cdc0ff600 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4ce18c8640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4009700 con 0x7f4cdc0ff600 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd400cd80 con 0x7f4cdc0ff600 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc1a0f80 con 0x7f4cdc0ff600 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4018860 con 0x7f4cdc0ff600 2024-01-25T16:05:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.980+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc1a13a0 con 0x7f4cdc0ff600 2024-01-25T16:05:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.981+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cdc10a350 con 0x7f4cdc0ff600 2024-01-25T16:05:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.982+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4cd40189c0 con 0x7f4cdc0ff600 2024-01-25T16:05:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.982+0000 7f4cd2ffd640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 0x7f4cc80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:32.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.983+0000 7f4ce10c7640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 0x7f4cc80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:32.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.983+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4cd4014070 con 0x7f4cdc0ff600 2024-01-25T16:05:32.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.983+0000 7f4ce10c7640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 0x7f4cc80a37a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc19a390 tx=0x7f4ccc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:32.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:32.987+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4cd4052600 con 0x7f4cdc0ff600 2024-01-25T16:05:33.055 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:32 smithi152 systemd[1]: Starting Ceph mds.i for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:33.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.207+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cdc10ea40 con 0x7f4cdc0ff600 2024-01-25T16:05:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.207+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/2836552340 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6871 (secure 0 0 0) 0x7f4cd40963a0 con 0x7f4cdc0ff600 2024-01-25T16:05:33.210 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:33.210 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9}],"filesystems":[]} 2024-01-25T16:05:33.210 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 msgr2=0x7f4cc80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 0x7f4cc80a37a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc19a390 tx=0x7f4ccc005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 msgr2=0x7f4cdc198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).stop 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 shutdown_connections 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cdc100900 0x7f4cdc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4cdc0fff60 0x7f4cdc1993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:33.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4cc80a12e0 0x7f4cc80a37a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 --2- 172.21.15.33:0/2836552340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cdc0ff600 0x7f4cdc198e80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 >> 172.21.15.33:0/2836552340 conn(0x7f4cdc0fb220 msgr2=0x7f4cdc0fcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 shutdown_connections 2024-01-25T16:05:33.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:33.209+0000 7f4ce3b2c640 1 -- 172.21.15.33:0/2836552340 wait complete. 2024-01-25T16:05:33.221 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:33 smithi033 ceph-mon[31417]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:33.335 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:33 smithi152 ceph-mon[36015]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:33.335 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 podman[66902]: 2024-01-25 16:05:33.008249782 +0000 UTC m=+0.018024725 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:33.335 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 podman[66902]: 2024-01-25 16:05:33.105343624 +0000 UTC m=+0.115118567 container create b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-01-25T16:05:33.335 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 podman[66902]: 2024-01-25 16:05:33.333223322 +0000 UTC m=+0.342998265 container init b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.31.3) 2024-01-25T16:05:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:33 smithi076 ceph-mon[35642]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:33.633 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 podman[66902]: 2024-01-25 16:05:33.335855197 +0000 UTC m=+0.345630131 container start b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:05:33.633 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 ceph-mds[66922]: -- 172.21.15.152:0/1073521091 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619e08af520 con 0x5619e0884c00 2024-01-25T16:05:33.633 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: starting mds.i at 2024-01-25T16:05:33.633 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 bash[66902]: b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 2024-01-25T16:05:33.633 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:05:33 smithi152 systemd[1]: Started Ceph mds.i for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2836552340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:34.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:34 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2836552340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:34.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:34 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2836552340' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:34.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:34.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:34.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:34 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:35.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:35.336 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:35 smithi033 ceph-mon[31417]: Deploying daemon mds.g on smithi033 2024-01-25T16:05:35.336 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:35 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:boot 2024-01-25T16:05:35.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:35 smithi033 ceph-mon[31417]: fsmap 9 up:standby 2024-01-25T16:05:35.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:35 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:05:35.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:35 smithi033 ceph-mon[31417]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/558684384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 msgr2=0x7fdbf80ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/558684384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf80ff7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009900 tx=0x7fdbe802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/558684384 shutdown_connections 2024-01-25T16:05:35.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/558684384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100f50 0x7fdbf81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/558684384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100590 0x7fdbf8100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/558684384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf80ff7c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/558684384 >> 172.21.15.33:0/558684384 conn(0x7fdbf80fa8d0 msgr2=0x7fdbf80fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:35.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/558684384 shutdown_connections 2024-01-25T16:05:35.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/558684384 wait complete. 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 Processor -- start 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.341+0000 7fdbfe5a9640 1 -- start start 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100590 0x7fdbf8196190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100f50 0x7fdbf819d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819d730 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819d8a0 con 0x7fdbf8100590 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf819da60 con 0x7fdbf8100f50 2024-01-25T16:05:35.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40882/0 (socket says 172.21.15.33:40882) 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 -- 172.21.15.33:0/3052863711 learned_addr learned my addr 172.21.15.33:0/3052863711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfcb46640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100f50 0x7fdbf819d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf77fe640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100590 0x7fdbf8196190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 -- 172.21.15.33:0/3052863711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100f50 msgr2=0x7fdbf819d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100f50 0x7fdbf819d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 -- 172.21.15.33:0/3052863711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100590 msgr2=0x7fdbf8196190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100590 0x7fdbf8196190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 -- 172.21.15.33:0/3052863711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe8009560 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf7fff640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009230 tx=0x7fdbe8004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80042d0 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbe8004430 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.342+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf819dce0 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.343+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8003d60 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.343+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf819e190 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.343+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf8106790 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.345+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdbe8005000 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.345+0000 7fdbf57fa640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 0x7fdbe40a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.345+0000 7fdbf77fe640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 0x7fdbe40a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:35.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.345+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbe80ef320 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.346+0000 7fdbf77fe640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 0x7fdbe40a3820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009b60 tx=0x7fdbe0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:35.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.349+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdbe80b8bb0 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:35 smithi152 ceph-mon[36015]: Deploying daemon mds.g on smithi033 2024-01-25T16:05:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:35 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:boot 2024-01-25T16:05:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:35 smithi152 ceph-mon[36015]: fsmap 9 up:standby 2024-01-25T16:05:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:35 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:05:35.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:35 smithi152 ceph-mon[36015]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:35 smithi076 ceph-mon[35642]: Deploying daemon mds.g on smithi033 2024-01-25T16:05:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:35 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:boot 2024-01-25T16:05:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:35 smithi076 ceph-mon[35642]: fsmap 9 up:standby 2024-01-25T16:05:35.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:35 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:05:35.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:35 smithi076 ceph-mon[35642]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.573+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbf810b6a0 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.574+0000 7fdbf57fa640 1 -- 172.21.15.33:0/3052863711 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7667 (secure 0 0 0) 0x7fdbe80baae0 con 0x7fdbf80ff3a0 2024-01-25T16:05:35.577 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:35.577 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-25T16:05:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 10 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 msgr2=0x7fdbe40a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 0x7fdbe40a3820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009b60 tx=0x7fdbe0005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 msgr2=0x7fdbf8195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8009230 tx=0x7fdbe8004dd0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 shutdown_connections 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbf8100f50 0x7fdbf819d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbf8100590 0x7fdbf8196190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdbe40a1360 0x7fdbe40a3820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 --2- 172.21.15.33:0/3052863711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbf80ff3a0 0x7fdbf8195c50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 >> 172.21.15.33:0/3052863711 conn(0x7fdbf80fa8d0 msgr2=0x7fdbf8103960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 shutdown_connections 2024-01-25T16:05:35.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:35.576+0000 7fdbfe5a9640 1 -- 172.21.15.33:0/3052863711 wait complete. 2024-01-25T16:05:36.005 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:35 smithi033 systemd[1]: Starting Ceph mds.g for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:36.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3052863711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:36.314 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 podman[72890]: 2024-01-25 16:05:35.905050141 +0000 UTC m=+0.017429868 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:36.315 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 podman[72890]: 2024-01-25 16:05:36.053863636 +0000 UTC m=+0.166243411 container create cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-01-25T16:05:36.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3052863711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:36.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3052863711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:36.814 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 podman[72890]: 2024-01-25 16:05:36.329705118 +0000 UTC m=+0.442084856 container init cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-01-25T16:05:36.814 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 podman[72890]: 2024-01-25 16:05:36.332455313 +0000 UTC m=+0.444835043 container start cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:05:36.814 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 ceph-mds[72909]: -- 172.21.15.33:0/3168145641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b59e69d520 con 0x55b59e672c00 2024-01-25T16:05:36.815 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g[72905]: starting mds.g at 2024-01-25T16:05:36.815 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 bash[72890]: cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf 2024-01-25T16:05:36.815 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:05:36 smithi033 systemd[1]: Started Ceph mds.g for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:37.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:37.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.379 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.380 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:37.380 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:37.380 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:37 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:37.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:37 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:37.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:37.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:37 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:37.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 -- 172.21.15.33:0/2135347999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a741017b0 msgr2=0x7f1a74103ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 --2- 172.21.15.33:0/2135347999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a741017b0 0x7f1a74103ba0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009900 tx=0x7f1a5c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 -- 172.21.15.33:0/2135347999 shutdown_connections 2024-01-25T16:05:37.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 --2- 172.21.15.33:0/2135347999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a74106a10 0x7f1a74108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 --2- 172.21.15.33:0/2135347999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741040e0 0x7f1a741064d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 --2- 172.21.15.33:0/2135347999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a741017b0 0x7f1a74103ba0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 -- 172.21.15.33:0/2135347999 >> 172.21.15.33:0/2135347999 conn(0x7f1a740fb1a0 msgr2=0x7f1a740fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 -- 172.21.15.33:0/2135347999 shutdown_connections 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.698+0000 7f1a7a366640 1 -- 172.21.15.33:0/2135347999 wait complete. 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 Processor -- start 2024-01-25T16:05:37.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 -- start start 2024-01-25T16:05:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741017b0 0x7f1a74196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a741040e0 0x7f1a74197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419e7d0 con 0x7f1a74106a10 2024-01-25T16:05:37.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419e940 con 0x7f1a741017b0 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a7a366640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7419eb00 con 0x7f1a741040e0 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a78903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a78903640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40912/0 (socket says 172.21.15.33:40912) 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a78903640 1 -- 172.21.15.33:0/750169563 learned_addr learned my addr 172.21.15.33:0/750169563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a737fe640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a741040e0 0x7f1a74197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.699+0000 7f1a73fff640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741017b0 0x7f1a74196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 -- 172.21.15.33:0/750169563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a741040e0 msgr2=0x7f1a74197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a741040e0 0x7f1a74197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 -- 172.21.15.33:0/750169563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741017b0 msgr2=0x7f1a74196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741017b0 0x7f1a74196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 -- 172.21.15.33:0/750169563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c009560 con 0x7f1a74106a10 2024-01-25T16:05:37.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a78903640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00eaa0 tx=0x7f1a6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c00cdb0 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a6c005020 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a7419ede0 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6c0116d0 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.700+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a7419f1b0 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.701+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a740fcc90 con 0x7f1a74106a10 2024-01-25T16:05:37.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.702+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f1a6c005190 con 0x7f1a74106a10 2024-01-25T16:05:37.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.702+0000 7f1a717fa640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 0x7f1a600a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:37.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.703+0000 7f1a73fff640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 0x7f1a600a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:37.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.703+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1a6c0cc8b0 con 0x7f1a74106a10 2024-01-25T16:05:37.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.703+0000 7f1a73fff640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 0x7f1a600a37a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c005ab0 tx=0x7f1a5c02c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:37.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.708+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1a6c0961f0 con 0x7f1a74106a10 2024-01-25T16:05:37.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.926+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a7410c5f0 con 0x7f1a74106a10 2024-01-25T16:05:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.926+0000 7f1a717fa640 1 -- 172.21.15.33:0/750169563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8459 (secure 0 0 0) 0x7f1a6c098120 con 0x7f1a74106a10 2024-01-25T16:05:37.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:37.929 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-25T16:05:37.929 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-01-25T16:05:37.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.928+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 msgr2=0x7f1a600a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:37.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.928+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 0x7f1a600a37a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c005ab0 tx=0x7f1a5c02c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.928+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 msgr2=0x7f1a7419e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.928+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c00eaa0 tx=0x7f1a6c00ef60 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 shutdown_connections 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1a600a12e0 0x7f1a600a37a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a74106a10 0x7f1a7419e260 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1a741040e0 0x7f1a74197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 --2- 172.21.15.33:0/750169563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a741017b0 0x7f1a74196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 >> 172.21.15.33:0/750169563 conn(0x7f1a740fb1a0 msgr2=0x7f1a74102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 shutdown_connections 2024-01-25T16:05:37.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:37.929+0000 7f1a7a366640 1 -- 172.21.15.33:0/750169563 wait complete. 2024-01-25T16:05:38.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:38 smithi033 ceph-mon[31417]: Deploying daemon mds.a on smithi033 2024-01-25T16:05:38.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:38 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.33:6838/3604277754,v1:172.21.15.33:6839/3604277754] up:boot 2024-01-25T16:05:38.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:38 smithi033 ceph-mon[31417]: fsmap 10 up:standby 2024-01-25T16:05:38.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:38 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:05:38.343 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:38 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/750169563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:38.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:38 smithi152 ceph-mon[36015]: Deploying daemon mds.a on smithi033 2024-01-25T16:05:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:38 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.33:6838/3604277754,v1:172.21.15.33:6839/3604277754] up:boot 2024-01-25T16:05:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:38 smithi152 ceph-mon[36015]: fsmap 10 up:standby 2024-01-25T16:05:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:38 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:05:38.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:38 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/750169563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:38 smithi076 ceph-mon[35642]: Deploying daemon mds.a on smithi033 2024-01-25T16:05:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:38 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.33:6838/3604277754,v1:172.21.15.33:6839/3604277754] up:boot 2024-01-25T16:05:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:38 smithi076 ceph-mon[35642]: fsmap 10 up:standby 2024-01-25T16:05:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:38 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:05:38.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:38 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/750169563' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:38.788 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:38 smithi033 systemd[1]: Starting Ceph mds.a for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:38 smithi033 podman[73250]: 2024-01-25 16:05:38.689016219 +0000 UTC m=+0.019439673 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:38 smithi033 podman[73250]: 2024-01-25 16:05:38.799980211 +0000 UTC m=+0.130403659 container create 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 podman[73250]: 2024-01-25 16:05:39.002306982 +0000 UTC m=+0.332730441 container init 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 podman[73250]: 2024-01-25 16:05:39.004836719 +0000 UTC m=+0.335260169 container start 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 ceph-mds[73269]: -- 172.21.15.33:0/2885300145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f9af12f520 con 0x55f9af104c00 2024-01-25T16:05:39.065 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a[73265]: starting mds.a at 2024-01-25T16:05:39.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:39 smithi152 ceph-mon[36015]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:39 smithi076 ceph-mon[35642]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:39 smithi033 ceph-mon[31417]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:39.565 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 bash[73250]: 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 2024-01-25T16:05:39.565 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:05:39 smithi033 systemd[1]: Started Ceph mds.a for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:39.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 -- 172.21.15.33:0/4089415517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181008a0 msgr2=0x7f5e18108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/4089415517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181008a0 0x7f5e18108140 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009cd0 tx=0x7f5e1002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 -- 172.21.15.33:0/4089415517 shutdown_connections 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/4089415517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181008a0 0x7f5e18108140 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/4089415517 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e180ffee0 0x7f5e18100360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/4089415517 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e181011b0 0x7f5e181015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 -- 172.21.15.33:0/4089415517 >> 172.21.15.33:0/4089415517 conn(0x7f5e180fa910 msgr2=0x7f5e180fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 -- 172.21.15.33:0/4089415517 shutdown_connections 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.061+0000 7f5e1e40f640 1 -- 172.21.15.33:0/4089415517 wait complete. 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 Processor -- start 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 -- start start 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e180ffee0 0x7f5e18196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e181008a0 0x7f5e18197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819e7f0 con 0x7f5e181011b0 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819e960 con 0x7f5e180ffee0 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.062+0000 7f5e1e40f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1819eb20 con 0x7f5e181008a0 2024-01-25T16:05:40.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:40.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54350/0 (socket says 172.21.15.33:54350) 2024-01-25T16:05:40.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 -- 172.21.15.33:0/348906900 learned_addr learned my addr 172.21.15.33:0/348906900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e17fff640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e180ffee0 0x7f5e18196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e177fe640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e181008a0 0x7f5e18197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 -- 172.21.15.33:0/348906900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e181008a0 msgr2=0x7f5e18197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e181008a0 0x7f5e18197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 -- 172.21.15.33:0/348906900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e180ffee0 msgr2=0x7f5e18196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e180ffee0 0x7f5e18196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 -- 172.21.15.33:0/348906900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009900 con 0x7f5e181011b0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1c9ac640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009ca0 tx=0x7f5e10032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10034c70 con 0x7f5e181011b0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e10003e30 con 0x7f5e181011b0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e1819eda0 con 0x7f5e181011b0 2024-01-25T16:05:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e10004870 con 0x7f5e181011b0 2024-01-25T16:05:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.063+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e1819f1a0 con 0x7f5e181011b0 2024-01-25T16:05:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.064+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e1810b500 con 0x7f5e181011b0 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.066+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f5e1003b040 con 0x7f5e181011b0 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.066+0000 7f5e157fa640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 0x7f5e000a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.066+0000 7f5e17fff640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 0x7f5e000a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.067+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e100eefc0 con 0x7f5e181011b0 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.067+0000 7f5e17fff640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 0x7f5e000a37a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5e181981e0 tx=0x7f5e08005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.070+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5e100b8850 con 0x7f5e181011b0 2024-01-25T16:05:40.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.290+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e1819f4e0 con 0x7f5e181011b0 2024-01-25T16:05:40.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.291+0000 7f5e157fa640 1 -- 172.21.15.33:0/348906900 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f5e100ba780 con 0x7f5e181011b0 2024-01-25T16:05:40.294 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:40.294 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-01-25T16:05:40.294 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-01-25T16:05:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 msgr2=0x7f5e000a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 0x7f5e000a37a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5e181981e0 tx=0x7f5e08005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 msgr2=0x7f5e1819e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009ca0 tx=0x7f5e10032790 comp rx=0 tx=0).stop 2024-01-25T16:05:40.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 shutdown_connections 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5e000a12e0 0x7f5e000a37a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e181011b0 0x7f5e1819e280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5e181008a0 0x7f5e18197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 --2- 172.21.15.33:0/348906900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e180ffee0 0x7f5e18196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.293+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 >> 172.21.15.33:0/348906900 conn(0x7f5e180fa910 msgr2=0x7f5e180fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.294+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 shutdown_connections 2024-01-25T16:05:40.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:40.294+0000 7f5e1e40f640 1 -- 172.21.15.33:0/348906900 wait complete. 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.33:6840/3585610078,v1:172.21.15.33:6841/3585610078] up:boot 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: fsmap 11 up:standby 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:40.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:40.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:40.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:40 smithi033 ceph-mon[31417]: Deploying daemon mds.b on smithi076 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.33:6840/3585610078,v1:172.21.15.33:6841/3585610078] up:boot 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: fsmap 11 up:standby 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:40.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:40 smithi152 ceph-mon[36015]: Deploying daemon mds.b on smithi076 2024-01-25T16:05:40.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.33:6840/3585610078,v1:172.21.15.33:6841/3585610078] up:boot 2024-01-25T16:05:40.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: fsmap 11 up:standby 2024-01-25T16:05:40.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:05:40.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:40.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:40 smithi076 ceph-mon[35642]: Deploying daemon mds.b on smithi076 2024-01-25T16:05:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:41 smithi152 ceph-mon[36015]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:41 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/348906900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:41.391 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:41 smithi076 ceph-mon[35642]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:41.391 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:41 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/348906900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:41.391 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 systemd[1]: Starting Ceph mds.b for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:05:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:41 smithi033 ceph-mon[31417]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:41 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/348906900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:41.668 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 podman[66577]: 2024-01-25 16:05:41.291838749 +0000 UTC m=+0.017782860 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:05:41.669 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 podman[66577]: 2024-01-25 16:05:41.397173354 +0000 UTC m=+0.123117446 container create 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3) 2024-01-25T16:05:41.669 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 podman[66577]: 2024-01-25 16:05:41.608828343 +0000 UTC m=+0.334772447 container init 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-01-25T16:05:41.669 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 podman[66577]: 2024-01-25 16:05:41.611338171 +0000 UTC m=+0.337282271 container start 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:05:41.669 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 ceph-mds[66596]: -- 172.21.15.76:0/1336850721 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560957d95520 con 0x560957d6a800 2024-01-25T16:05:41.669 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: starting mds.b at 2024-01-25T16:05:42.168 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 bash[66577]: 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f 2024-01-25T16:05:42.168 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:41 smithi076 systemd[1]: Started Ceph mds.b for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:05:42.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 -- 172.21.15.33:0/2000756523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 msgr2=0x7faf8c0ff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 --2- 172.21.15.33:0/2000756523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c0ff420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009900 tx=0x7faf7c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 -- 172.21.15.33:0/2000756523 shutdown_connections 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 --2- 172.21.15.33:0/2000756523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c102990 0x7faf8c109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 --2- 172.21.15.33:0/2000756523 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c101fd0 0x7faf8c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 --2- 172.21.15.33:0/2000756523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c0ff420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 -- 172.21.15.33:0/2000756523 >> 172.21.15.33:0/2000756523 conn(0x7faf8c0fa930 msgr2=0x7faf8c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 -- 172.21.15.33:0/2000756523 shutdown_connections 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.411+0000 7faf912e7640 1 -- 172.21.15.33:0/2000756523 wait complete. 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 Processor -- start 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 -- start start 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c101fd0 0x7faf8c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c102990 0x7faf8c19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19e820 con 0x7faf8c0ff000 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19e990 con 0x7faf8c101fd0 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf912e7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c19eb50 con 0x7faf8c102990 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf8affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf8affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54360/0 (socket says 172.21.15.33:54360) 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf8affd640 1 -- 172.21.15.33:0/1420769763 learned_addr learned my addr 172.21.15.33:0/1420769763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf83fff640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c101fd0 0x7faf8c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.412+0000 7faf8b7fe640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c102990 0x7faf8c19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 -- 172.21.15.33:0/1420769763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c102990 msgr2=0x7faf8c19e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c102990 0x7faf8c19e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 -- 172.21.15.33:0/1420769763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c101fd0 msgr2=0x7faf8c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c101fd0 0x7faf8c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 -- 172.21.15.33:0/1420769763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf84009840 con 0x7faf8c0ff000 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf8affd640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009230 tx=0x7faf7c004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0043d0 con 0x7faf8c0ff000 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf7c0054c0 con 0x7faf8c0ff000 2024-01-25T16:05:42.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c009560 con 0x7faf8c0ff000 2024-01-25T16:05:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c003d40 con 0x7faf8c0ff000 2024-01-25T16:05:42.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.413+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c19f150 con 0x7faf8c0ff000 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.414+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c1081a0 con 0x7faf8c0ff000 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.415+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faf7c005000 con 0x7faf8c0ff000 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.416+0000 7faf88ff9640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 0x7faf780a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.416+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf7c0eefd0 con 0x7faf8c0ff000 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.416+0000 7faf83fff640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 0x7faf780a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.416+0000 7faf83fff640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 0x7faf780a37a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf74005d00 tx=0x7faf7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:42.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.420+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faf7c0b8860 con 0x7faf8c0ff000 2024-01-25T16:05:42.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.665+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf8c10c830 con 0x7faf8c0ff000 2024-01-25T16:05:42.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.666+0000 7faf88ff9640 1 -- 172.21.15.33:0/1420769763 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10040 (secure 0 0 0) 0x7faf7c0ba790 con 0x7faf8c0ff000 2024-01-25T16:05:42.668 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:42.668 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24484,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24494,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24517,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24536,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-01-25T16:05:42.669 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-01-25T16:05:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 msgr2=0x7faf780a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 0x7faf780a37a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf74005d00 tx=0x7faf7400b410 comp rx=0 tx=0).stop 2024-01-25T16:05:42.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 msgr2=0x7faf8c196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009230 tx=0x7faf7c004ed0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 shutdown_connections 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faf8c102990 0x7faf8c19e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf8c101fd0 0x7faf8c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faf780a12e0 0x7faf780a37a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 --2- 172.21.15.33:0/1420769763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf8c0ff000 0x7faf8c196d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.667+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 >> 172.21.15.33:0/1420769763 conn(0x7faf8c0fa930 msgr2=0x7faf8c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.668+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 shutdown_connections 2024-01-25T16:05:42.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.668+0000 7faf912e7640 1 -- 172.21.15.33:0/1420769763 wait complete. 2024-01-25T16:05:42.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:05:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 -- 172.21.15.33:0/2222708602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 msgr2=0x7f2520100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 --2- 172.21.15.33:0/2222708602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520100040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2518009ca0 tx=0x7f251802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 -- 172.21.15.33:0/2222708602 shutdown_connections 2024-01-25T16:05:42.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 --2- 172.21.15.33:0/2222708602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25201017d0 0x7f2520108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 --2- 172.21.15.33:0/2222708602 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2520100e10 0x7f2520101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 --2- 172.21.15.33:0/2222708602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520100040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.782+0000 7f252715a640 1 -- 172.21.15.33:0/2222708602 >> 172.21.15.33:0/2222708602 conn(0x7f25200fb1d0 msgr2=0x7f25200fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- 172.21.15.33:0/2222708602 shutdown_connections 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- 172.21.15.33:0/2222708602 wait complete. 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 Processor -- start 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- start start 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2520100e10 0x7f2520199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25201017d0 0x7f25201a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25201a0a40 con 0x7f25200ffc20 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25201a0bb0 con 0x7f2520100e10 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f252715a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25201a0d70 con 0x7f25201017d0 2024-01-25T16:05:42.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f2524ef6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f2524ef6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54370/0 (socket says 172.21.15.33:54370) 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f2524ef6640 1 -- 172.21.15.33:0/2299191820 learned_addr learned my addr 172.21.15.33:0/2299191820 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f25256f7640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25201017d0 0x7f25201a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.783+0000 7f251ffff640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2520100e10 0x7f2520199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 -- 172.21.15.33:0/2299191820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25201017d0 msgr2=0x7f25201a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25201017d0 0x7f25201a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 -- 172.21.15.33:0/2299191820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2520100e10 msgr2=0x7f2520199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2520100e10 0x7f2520199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 -- 172.21.15.33:0/2299191820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518009900 con 0x7f25200ffc20 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f2524ef6640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2518037040 tx=0x7f2518032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518034c70 con 0x7f25200ffc20 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2518003e30 con 0x7f25200ffc20 2024-01-25T16:05:42.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25201a0ff0 con 0x7f25200ffc20 2024-01-25T16:05:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251803d3f0 con 0x7f25200ffc20 2024-01-25T16:05:42.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.784+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25201a13c0 con 0x7f25200ffc20 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.785+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252010c610 con 0x7f25200ffc20 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.786+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f251803b040 con 0x7f25200ffc20 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.787+0000 7f251dffb640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 0x7f25080a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.787+0000 7f251ffff640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 0x7f25080a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.787+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f25180eec80 con 0x7f25200ffc20 2024-01-25T16:05:42.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.787+0000 7f251ffff640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 0x7f25080a37a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f252019a460 tx=0x7f2514005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:42.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.791+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f25180b85b0 con 0x7f25200ffc20 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:boot 2024-01-25T16:05:42.965 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: fsmap 12 up:standby 2024-01-25T16:05:42.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:05:42.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:42.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1420769763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:42.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.973+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f252010c7a0 con 0x7f25200ffc20 2024-01-25T16:05:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.974+0000 7f251dffb640 1 -- 172.21.15.33:0/2299191820 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18327 (secure 0 0 0) 0x7f25180bbbd0 con 0x7f25200ffc20 2024-01-25T16:05:42.981 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":82,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:04:56.833580+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 msgr2=0x7f25080a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 0x7f25080a37a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f252019a460 tx=0x7f2514005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 msgr2=0x7f2520198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2518037040 tx=0x7f2518032810 comp rx=0 tx=0).stop 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 shutdown_connections 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25201017d0 0x7f25201a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2520100e10 0x7f2520199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f25080a12e0 0x7f25080a37a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 --2- 172.21.15.33:0/2299191820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25200ffc20 0x7f2520198f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 >> 172.21.15.33:0/2299191820 conn(0x7f25200fb1d0 msgr2=0x7f2520108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 shutdown_connections 2024-01-25T16:05:42.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:42.977+0000 7f252715a640 1 -- 172.21.15.33:0/2299191820 wait complete. 2024-01-25T16:05:42.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:05:42.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:05:43.030 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:boot 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: fsmap 12 up:standby 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:43.031 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1420769763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:43.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3427356425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc100de0 msgr2=0x7fe4cc101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3427356425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc100de0 0x7fe4cc101260 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009900 tx=0x7fe4c002bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3427356425 shutdown_connections 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3427356425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3427356425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc100de0 0x7fe4cc101260 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3427356425 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 0x7fe4cc100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3427356425 >> 172.21.15.33:0/3427356425 conn(0x7fe4cc0fb180 msgr2=0x7fe4cc0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3427356425 shutdown_connections 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3427356425 wait complete. 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.097+0000 7fe4d1dbe640 1 Processor -- start 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 -- start start 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 0x7fe4cc195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc100de0 0x7fe4cc196130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19d6e0 con 0x7fe4cc1017a0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19d850 con 0x7fe4cc100de0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19da10 con 0x7fe4cc0ffbf0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54384/0 (socket says 172.21.15.33:54384) 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 -- 172.21.15.33:0/3698135138 learned_addr learned my addr 172.21.15.33:0/3698135138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cb7fe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 0x7fe4cc195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4caffd640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc100de0 0x7fe4cc196130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 -- 172.21.15.33:0/3698135138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 msgr2=0x7fe4cc195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 0x7fe4cc195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 -- 172.21.15.33:0/3698135138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc100de0 msgr2=0x7fe4cc196130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc100de0 0x7fe4cc196130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 -- 172.21.15.33:0/3698135138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c0009560 con 0x7fe4cc1017a0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4cbfff640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eb10 tx=0x7fe4c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4004b20 con 0x7fe4cc1017a0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.098+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc19dcf0 con 0x7fe4cc1017a0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.099+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c4004c80 con 0x7fe4cc1017a0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.099+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4010690 con 0x7fe4cc1017a0 2024-01-25T16:05:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.099+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc19e1d0 con 0x7fe4cc1017a0 2024-01-25T16:05:43.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.100+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc106fb0 con 0x7fe4cc1017a0 2024-01-25T16:05:43.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.101+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe4c4005020 con 0x7fe4cc1017a0 2024-01-25T16:05:43.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.101+0000 7fe4c8ff9640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 0x7fe4b40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.101+0000 7fe4cb7fe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 0x7fe4b40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.101+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe4c4014070 con 0x7fe4cc1017a0 2024-01-25T16:05:43.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.102+0000 7fe4cb7fe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 0x7fe4b40a37a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc100c40 tx=0x7fe4bc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.105+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe4c4096530 con 0x7fe4cc1017a0 2024-01-25T16:05:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:boot 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: fsmap 12 up:standby 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:43.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1420769763' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.290+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4cc10b6b0 con 0x7fe4cc1017a0 2024-01-25T16:05:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.290+0000 7fe4c8ff9640 1 -- 172.21.15.33:0/3698135138 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4c4099b50 con 0x7fe4cc1017a0 2024-01-25T16:05:43.293 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:05:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.292+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 msgr2=0x7fe4b40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 0x7fe4b40a37a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc100c40 tx=0x7fe4bc005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 msgr2=0x7fe4cc19d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400eb10 tx=0x7fe4c400c2d0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 shutdown_connections 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4b40a12e0 0x7fe4b40a37a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4cc1017a0 0x7fe4cc19d1a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4cc100de0 0x7fe4cc196130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 --2- 172.21.15.33:0/3698135138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4cc0ffbf0 0x7fe4cc195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 >> 172.21.15.33:0/3698135138 conn(0x7fe4cc0fb180 msgr2=0x7fe4cc104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 shutdown_connections 2024-01-25T16:05:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.293+0000 7fe4d1dbe640 1 -- 172.21.15.33:0/3698135138 wait complete. 2024-01-25T16:05:43.307 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-25T16:05:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/1995257493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f41008e0 msgr2=0x7fb9f4100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/1995257493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f41008e0 0x7fb9f4100d60 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009ca0 tx=0x7fb9ec02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/1995257493 shutdown_connections 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/1995257493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f41008e0 0x7fb9f4100d60 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/1995257493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 0x7fb9f41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/1995257493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f40ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.412+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/1995257493 >> 172.21.15.33:0/1995257493 conn(0x7fb9f40fb180 msgr2=0x7fb9f40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/1995257493 shutdown_connections 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/1995257493 wait complete. 2024-01-25T16:05:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 Processor -- start 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- start start 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 0x7fb9f4199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f41008e0 0x7fb9f41a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41a0930 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41a0aa0 con 0x7fb9f41008e0 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9fbbdc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f41a0c60 con 0x7fb9f40fff40 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.413+0000 7fb9f9978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54402/0 (socket says 172.21.15.33:54402) 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 -- 172.21.15.33:0/3325723561 learned_addr learned my addr 172.21.15.33:0/3325723561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:43.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9fa179640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f41008e0 0x7fb9f41a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9177640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 0x7fb9f4199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 -- 172.21.15.33:0/3325723561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 msgr2=0x7fb9f4199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 0x7fb9f4199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 -- 172.21.15.33:0/3325723561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f41008e0 msgr2=0x7fb9f41a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f41008e0 0x7fb9f41a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 -- 172.21.15.33:0/3325723561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4009490 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9f9978640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e9f0 tx=0x7fb9f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0009700 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec009900 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f41a1230 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9f000cd80 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.414+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0018840 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.416+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f410a510 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.416+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb9f0010af0 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.417+0000 7fb9eaffd640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 0x7fb9e00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.417+0000 7fb9f9177640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 0x7fb9e00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.417+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb9f0014070 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.417+0000 7fb9f9177640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 0x7fb9e00a37a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9f419a350 tx=0x7fb9e4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.422+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb9f0052600 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.640+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb9f410e9b0 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.640+0000 7fb9eaffd640 1 -- 172.21.15.33:0/3325723561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb9f00963a0 con 0x7fb9f40ff5e0 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:05:43.643 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 msgr2=0x7fb9e00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 0x7fb9e00a37a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9f419a350 tx=0x7fb9e4005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 msgr2=0x7fb9f4198e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000e9f0 tx=0x7fb9f000eeb0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 shutdown_connections 2024-01-25T16:05:43.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9f41008e0 0x7fb9f41a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9f40fff40 0x7fb9f4199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9e00a12e0 0x7fb9e00a37a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 --2- 172.21.15.33:0/3325723561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9f40ff5e0 0x7fb9f4198e40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.642+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 >> 172.21.15.33:0/3325723561 conn(0x7fb9f40fb180 msgr2=0x7fb9f40fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.643+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 shutdown_connections 2024-01-25T16:05:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.643+0000 7fb9fbbdc640 1 -- 172.21.15.33:0/3325723561 wait complete. 2024-01-25T16:05:43.658 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-25T16:05:43.659 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-25T16:05:43.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 -- 172.21.15.33:0/3324305399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff8100d20 msgr2=0x7feff8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 --2- 172.21.15.33:0/3324305399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff8100d20 0x7feff8101140 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fefe8009900 tx=0x7fefe802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 -- 172.21.15.33:0/3324305399 shutdown_connections 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 --2- 172.21.15.33:0/3324305399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff81028d0 0x7feff8109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 --2- 172.21.15.33:0/3324305399 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 0x7feff8102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 --2- 172.21.15.33:0/3324305399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff8100d20 0x7feff8101140 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 -- 172.21.15.33:0/3324305399 >> 172.21.15.33:0/3324305399 conn(0x7feff80fc290 msgr2=0x7feff80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 -- 172.21.15.33:0/3324305399 shutdown_connections 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.767+0000 7feffe1de640 1 -- 172.21.15.33:0/3324305399 wait complete. 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 Processor -- start 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 -- start start 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff8100d20 0x7feff8196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 0x7feff8197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819e810 con 0x7feff81028d0 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819e980 con 0x7feff8100d20 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feffe1de640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff819eb40 con 0x7feff8101f10 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54422/0 (socket says 172.21.15.33:54422) 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff7fff640 1 -- 172.21.15.33:0/3237565636 learned_addr learned my addr 172.21.15.33:0/3237565636 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff6ffd640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 0x7feff8197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff77fe640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff8100d20 0x7feff8196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff7fff640 1 -- 172.21.15.33:0/3237565636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 msgr2=0x7feff8197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.768+0000 7feff7fff640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 0x7feff8197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff7fff640 1 -- 172.21.15.33:0/3237565636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff8100d20 msgr2=0x7feff8196d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff7fff640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff8100d20 0x7feff8196d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff7fff640 1 -- 172.21.15.33:0/3237565636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe8009560 con 0x7feff81028d0 2024-01-25T16:05:43.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff7fff640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feff000eb10 tx=0x7feff000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0004b20 con 0x7feff81028d0 2024-01-25T16:05:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff0004c80 con 0x7feff81028d0 2024-01-25T16:05:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff819ee20 con 0x7feff81028d0 2024-01-25T16:05:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0010690 con 0x7feff81028d0 2024-01-25T16:05:43.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.769+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff819f1f0 con 0x7feff81028d0 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.770+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8108150 con 0x7feff81028d0 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.771+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7feff0005020 con 0x7feff81028d0 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.772+0000 7feff4ff9640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 0x7fefe00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.772+0000 7feff77fe640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 0x7fefe00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.772+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feff0014070 con 0x7feff81028d0 2024-01-25T16:05:43.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.772+0000 7feff77fe640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 0x7fefe00a37a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feff8198200 tx=0x7fefe802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:43.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.777+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7feff0096600 con 0x7feff81028d0 2024-01-25T16:05:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:43.957+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feff810c5b0 con 0x7feff81028d0 2024-01-25T16:05:44.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2299191820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:44.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3698135138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:44.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3325723561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-25T16:05:44.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:43 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.132 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2299191820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3698135138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3325723561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:43 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2299191820' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3698135138' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3325723561' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-01-25T16:05:44.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:43 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.746+0000 7feff4ff9640 1 -- 172.21.15.33:0/3237565636 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7feff810c5b0 con 0x7feff81028d0 2024-01-25T16:05:44.749 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.748+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 msgr2=0x7fefe00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.748+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 0x7fefe00a37a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feff8198200 tx=0x7fefe802c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.748+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 msgr2=0x7feff819e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.748+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7feff000eb10 tx=0x7feff000c2d0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 shutdown_connections 2024-01-25T16:05:44.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fefe00a12e0 0x7fefe00a37a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feff81028d0 0x7feff819e2a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feff8101f10 0x7feff8197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 --2- 172.21.15.33:0/3237565636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feff8100d20 0x7feff8196d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 >> 172.21.15.33:0/3237565636 conn(0x7feff80fc290 msgr2=0x7feff8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 shutdown_connections 2024-01-25T16:05:44.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.749+0000 7feffe1de640 1 -- 172.21.15.33:0/3237565636 wait complete. 2024-01-25T16:05:44.767 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-25T16:05:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 -- 172.21.15.33:0/3355280299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 msgr2=0x7f0ca0108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 --2- 172.21.15.33:0/3355280299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca0108030 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009ca0 tx=0x7f0c9802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 -- 172.21.15.33:0/3355280299 shutdown_connections 2024-01-25T16:05:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 --2- 172.21.15.33:0/3355280299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca0108030 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 --2- 172.21.15.33:0/3355280299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 0x7f0ca0101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 --2- 172.21.15.33:0/3355280299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 0x7f0ca0100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 -- 172.21.15.33:0/3355280299 >> 172.21.15.33:0/3355280299 conn(0x7f0ca00fb180 msgr2=0x7f0ca00fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 -- 172.21.15.33:0/3355280299 shutdown_connections 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.875+0000 7f0ca509b640 1 -- 172.21.15.33:0/3355280299 wait complete. 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 Processor -- start 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 -- start start 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 0x7f0ca0196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 0x7f0ca0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca019e6c0 con 0x7f0ca01017a0 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca019e830 con 0x7f0ca00ffbf0 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0ca509b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca019e9f0 con 0x7f0ca0100de0 2024-01-25T16:05:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.876+0000 7f0c9f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54446/0 (socket says 172.21.15.33:54446) 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 -- 172.21.15.33:0/99777666 learned_addr learned my addr 172.21.15.33:0/99777666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9ed9d640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 0x7f0ca0196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9e59c640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 0x7f0ca0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 -- 172.21.15.33:0/99777666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 msgr2=0x7f0ca0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 0x7f0ca0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 -- 172.21.15.33:0/99777666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 msgr2=0x7f0ca0196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 0x7f0ca0196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 -- 172.21.15.33:0/99777666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98009900 con 0x7f0ca01017a0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c9f59e640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c98037040 tx=0x7f0c98032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98034c70 con 0x7f0ca01017a0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca019ec70 con 0x7f0ca01017a0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c98003e30 con 0x7f0ca01017a0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c98004870 con 0x7f0ca01017a0 2024-01-25T16:05:44.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.877+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca019f120 con 0x7f0ca01017a0 2024-01-25T16:05:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.878+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca0106f10 con 0x7f0ca01017a0 2024-01-25T16:05:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.879+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0c9803b040 con 0x7f0ca01017a0 2024-01-25T16:05:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.880+0000 7f0c87fff640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 0x7f0c880a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.880+0000 7f0c9ed9d640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 0x7f0c880a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.880+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f0c980ef1c0 con 0x7f0ca01017a0 2024-01-25T16:05:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.880+0000 7f0c9ed9d640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 0x7f0c880a37a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0100c40 tx=0x7f0c9000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:44.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:44.884+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0c980b8970 con 0x7f0ca01017a0 2024-01-25T16:05:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:44.918 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:44.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:44.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-25T16:05:44.919 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:44 smithi076 ceph-mon[35642]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:45.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-25T16:05:45.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:44 smithi033 ceph-mon[31417]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.064+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0ca010c5e0 con 0x7f0ca01017a0 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-01-25T16:05:45.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:44 smithi152 ceph-mon[36015]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.750+0000 7f0c87fff640 1 -- 172.21.15.33:0/99777666 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0c980bbf90 con 0x7f0ca01017a0 2024-01-25T16:05:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-01-25T16:05:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 msgr2=0x7f0c880a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 0x7f0c880a37a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0100c40 tx=0x7f0c9000b390 comp rx=0 tx=0).stop 2024-01-25T16:05:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 msgr2=0x7f0ca019e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c98037040 tx=0x7f0c98032810 comp rx=0 tx=0).stop 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 shutdown_connections 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0c880a12e0 0x7f0c880a37a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ca01017a0 0x7f0ca019e180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ca0100de0 0x7f0ca0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 --2- 172.21.15.33:0/99777666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ca00ffbf0 0x7f0ca0196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.753+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 >> 172.21.15.33:0/99777666 conn(0x7f0ca00fb180 msgr2=0x7f0ca01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.754+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 shutdown_connections 2024-01-25T16:05:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.754+0000 7f0ca509b640 1 -- 172.21.15.33:0/99777666 wait complete. 2024-01-25T16:05:45.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 -- 172.21.15.33:0/541110594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc102960 msgr2=0x7f09fc1091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 --2- 172.21.15.33:0/541110594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc102960 0x7f09fc1091f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009ca0 tx=0x7f09ec02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 -- 172.21.15.33:0/541110594 shutdown_connections 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 --2- 172.21.15.33:0/541110594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc102960 0x7f09fc1091f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 --2- 172.21.15.33:0/541110594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc101fa0 0x7f09fc102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 --2- 172.21.15.33:0/541110594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc0fefd0 0x7f09fc0ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 -- 172.21.15.33:0/541110594 >> 172.21.15.33:0/541110594 conn(0x7f09fc0fa920 msgr2=0x7f09fc0fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 -- 172.21.15.33:0/541110594 shutdown_connections 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.879+0000 7f0a014a8640 1 -- 172.21.15.33:0/541110594 wait complete. 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 Processor -- start 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 -- start start 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc0fefd0 0x7f09fc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc102960 0x7f09fc1a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0920 con 0x7f09fc101fa0 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0a90 con 0x7f09fc102960 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f0a014a8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09fc1a0c50 con 0x7f09fc0fefd0 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f09fa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f09fa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54468/0 (socket says 172.21.15.33:54468) 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f09fa7fc640 1 -- 172.21.15.33:0/4079460077 learned_addr learned my addr 172.21.15.33:0/4079460077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.880+0000 7f09fb7fe640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc102960 0x7f09fc1a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09faffd640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc0fefd0 0x7f09fc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 -- 172.21.15.33:0/4079460077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc0fefd0 msgr2=0x7f09fc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc0fefd0 0x7f09fc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 -- 172.21.15.33:0/4079460077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc102960 msgr2=0x7f09fc1a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc102960 0x7f09fc1a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 -- 172.21.15.33:0/4079460077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009900 con 0x7f09fc101fa0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09fa7fc640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f09f000d770 tx=0x7f09f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0009c30 con 0x7f09fc101fa0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f000de80 con 0x7f09fc101fa0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc1a0f30 con 0x7f09fc101fa0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f0004de0 con 0x7f09fc101fa0 2024-01-25T16:05:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.881+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc1a1360 con 0x7f09fc101fa0 2024-01-25T16:05:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.882+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09fc10c5e0 con 0x7f09fc101fa0 2024-01-25T16:05:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.883+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f09f0004f40 con 0x7f09fc101fa0 2024-01-25T16:05:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.884+0000 7f09e3fff640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 0x7f09e40a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.884+0000 7f09faffd640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 0x7f09e40a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.884+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f09f00cc7d0 con 0x7f09fc101fa0 2024-01-25T16:05:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.884+0000 7f09faffd640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 0x7f09e40a37a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005d00 tx=0x7f09f4005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:45.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:45.888+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f09f0095d60 con 0x7f09fc101fa0 2024-01-25T16:05:46.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-25T16:05:46.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: osdmap e83: 12 total, 12 up, 12 in 2024-01-25T16:05:46.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-25T16:05:46.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:46.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:45 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.106+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f09fc1a1550 con 0x7f09fc101fa0 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: osdmap e83: 12 total, 12 up, 12 in 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:46.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:46.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:45 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3237565636' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: osdmap e83: 12 total, 12 up, 12 in 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:05:46.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:45 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:05:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.781+0000 7f09e3fff640 1 -- 172.21.15.33:0/4079460077 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f09f00170c0 con 0x7f09fc101fa0 2024-01-25T16:05:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 msgr2=0x7f09e40a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 0x7f09e40a37a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f09f4005d00 tx=0x7f09f4005c50 comp rx=0 tx=0).stop 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 msgr2=0x7f09fc199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f09f000d770 tx=0x7f09f000dc30 comp rx=0 tx=0).stop 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 shutdown_connections 2024-01-25T16:05:46.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f09fc102960 0x7f09fc1a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f09e40a12e0 0x7f09e40a37a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09fc101fa0 0x7f09fc199370 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f09f000d770 tx=0x7f09f000dc30 comp rx=0 tx=0).stop 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 --2- 172.21.15.33:0/4079460077 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09fc0fefd0 0x7f09fc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.785+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 >> 172.21.15.33:0/4079460077 conn(0x7f09fc0fa920 msgr2=0x7f09fc0ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.786+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 shutdown_connections 2024-01-25T16:05:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.786+0000 7f0a014a8640 1 -- 172.21.15.33:0/4079460077 wait complete. 2024-01-25T16:05:46.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:05:46.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 -- 172.21.15.33:0/404562226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574104120 msgr2=0x7fb574106510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:46.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 --2- 172.21.15.33:0/404562226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574104120 0x7fb574106510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009900 tx=0x7fb55c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:46.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 -- 172.21.15.33:0/404562226 shutdown_connections 2024-01-25T16:05:46.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 --2- 172.21.15.33:0/404562226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574106a50 0x7fb574108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 --2- 172.21.15.33:0/404562226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574104120 0x7fb574106510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 --2- 172.21.15.33:0/404562226 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 -- 172.21.15.33:0/404562226 >> 172.21.15.33:0/404562226 conn(0x7fb5740fb180 msgr2=0x7fb5740fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 -- 172.21.15.33:0/404562226 shutdown_connections 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.915+0000 7fb57959e640 1 -- 172.21.15.33:0/404562226 wait complete. 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 Processor -- start 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 -- start start 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 0x7fb574197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419e830 con 0x7fb574106a50 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419e9a0 con 0x7fb574104120 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb57959e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb57419eb60 con 0x7fb5741017f0 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54490/0 (socket says 172.21.15.33:54490) 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 -- 172.21.15.33:0/1221094918 learned_addr learned my addr 172.21.15.33:0/1221094918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb572ffd640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb56bfff640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 0x7fb574197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 -- 172.21.15.33:0/1221094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 msgr2=0x7fb574196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 -- 172.21.15.33:0/1221094918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 msgr2=0x7fb574197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 0x7fb574197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb5737fe640 1 -- 172.21.15.33:0/1221094918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c009560 con 0x7fb574106a50 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb572ffd640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.916+0000 7fb56bfff640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 0x7fb574197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb5737fe640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00cad0 tx=0x7fb56c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00ae20 con 0x7fb574106a50 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57419ee40 con 0x7fb574106a50 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb56c005020 con 0x7fb574106a50 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00f6b0 con 0x7fb574106a50 2024-01-25T16:05:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.917+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57419f270 con 0x7fb574106a50 2024-01-25T16:05:46.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.918+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5740fcc70 con 0x7fb574106a50 2024-01-25T16:05:46.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.919+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fb56c005190 con 0x7fb574106a50 2024-01-25T16:05:46.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.919+0000 7fb570ff9640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 0x7fb5600a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:46.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.920+0000 7fb572ffd640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 0x7fb5600a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:46.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.920+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb56c0ccc10 con 0x7fb574106a50 2024-01-25T16:05:46.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.920+0000 7fb572ffd640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 0x7fb5600a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5640058c0 tx=0x7fb564005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:46.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:46.924+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb56c096140 con 0x7fb574106a50 2024-01-25T16:05:47.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:46 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-25T16:05:47.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:46 smithi033 ceph-mon[31417]: osdmap e84: 12 total, 12 up, 12 in 2024-01-25T16:05:47.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:46 smithi033 ceph-mon[31417]: pgmap v283: 97 pgs: 6 creating+peering, 90 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:47.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:46 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-25T16:05:47.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:46 smithi033 ceph-mon[31417]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-25T16:05:47.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.106+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb57402cb70 con 0x7fb574106a50 2024-01-25T16:05:47.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.107+0000 7fb570ff9640 1 -- 172.21.15.33:0/1221094918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22037 (secure 0 0 0) 0x7fb56c01d090 con 0x7fb574106a50 2024-01-25T16:05:47.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:47.110 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:05:46.107586+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:47.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 msgr2=0x7fb5600a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 0x7fb5600a37a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb5640058c0 tx=0x7fb564005d30 comp rx=0 tx=0).stop 2024-01-25T16:05:47.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 msgr2=0x7fb57419e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00cad0 tx=0x7fb56c00cf90 comp rx=0 tx=0).stop 2024-01-25T16:05:47.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 shutdown_connections 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb5600a12e0 0x7fb5600a37a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb574106a50 0x7fb57419e2f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb574104120 0x7fb574197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 --2- 172.21.15.33:0/1221094918 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb5741017f0 0x7fb574196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 >> 172.21.15.33:0/1221094918 conn(0x7fb5740fb180 msgr2=0x7fb574102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 shutdown_connections 2024-01-25T16:05:47.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.109+0000 7fb57959e640 1 -- 172.21.15.33:0/1221094918 wait complete. 2024-01-25T16:05:47.123 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:05:47.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:46 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-25T16:05:47.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:46 smithi152 ceph-mon[36015]: osdmap e84: 12 total, 12 up, 12 in 2024-01-25T16:05:47.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:46 smithi152 ceph-mon[36015]: pgmap v283: 97 pgs: 6 creating+peering, 90 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:47.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:46 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-25T16:05:47.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:46 smithi152 ceph-mon[36015]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-25T16:05:47.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:46 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/99777666' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-01-25T16:05:47.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:46 smithi076 ceph-mon[35642]: osdmap e84: 12 total, 12 up, 12 in 2024-01-25T16:05:47.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:46 smithi076 ceph-mon[35642]: pgmap v283: 97 pgs: 6 creating+peering, 90 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:47.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:46 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-01-25T16:05:47.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:46 smithi076 ceph-mon[35642]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-01-25T16:05:47.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/573604116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 msgr2=0x7fd2cc108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/573604116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc108050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009ca0 tx=0x7fd2c402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/573604116 shutdown_connections 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/573604116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc108050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/573604116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc100e00 0x7fd2cc101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/573604116 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc0ffc10 0x7fd2cc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/573604116 >> 172.21.15.33:0/573604116 conn(0x7fd2cc0fb180 msgr2=0x7fd2cc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/573604116 shutdown_connections 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/573604116 wait complete. 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 Processor -- start 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.223+0000 7fd2d3b5a640 1 -- start start 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc0ffc10 0x7fd2cc069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100e00 0x7fd2cc19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19e6e0 con 0x7fd2cc1017c0 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19e850 con 0x7fd2cc0ffc10 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19ea10 con 0x7fd2cc100e00 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54510/0 (socket says 172.21.15.33:54510) 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 -- 172.21.15.33:0/1698290490 learned_addr learned my addr 172.21.15.33:0/1698290490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d18f6640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc0ffc10 0x7fd2cc069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d10f5640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100e00 0x7fd2cc19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 -- 172.21.15.33:0/1698290490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100e00 msgr2=0x7fd2cc19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100e00 0x7fd2cc19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 -- 172.21.15.33:0/1698290490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc0ffc10 msgr2=0x7fd2cc069ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc0ffc10 0x7fd2cc069ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 -- 172.21.15.33:0/1698290490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009900 con 0x7fd2cc1017c0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d20f7640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4037040 tx=0x7fd2c4032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4034e80 con 0x7fd2cc1017c0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2c4003e30 con 0x7fd2cc1017c0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc19ec90 con 0x7fd2cc1017c0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c403d3f0 con 0x7fd2cc1017c0 2024-01-25T16:05:47.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.224+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc19f140 con 0x7fd2cc1017c0 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.225+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc106f30 con 0x7fd2cc1017c0 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.226+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd2c403b040 con 0x7fd2cc1017c0 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.227+0000 7fd2c2ffd640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.227+0000 7fd2d18f6640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.227+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2c40ef670 con 0x7fd2cc1017c0 2024-01-25T16:05:47.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.227+0000 7fd2d18f6640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80058c0 tx=0x7fd2c8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.230+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd2c40b8c40 con 0x7fd2cc1017c0 2024-01-25T16:05:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.413+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2cc02cb10 con 0x7fd2cc1017c0 2024-01-25T16:05:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.415+0000 7fd2c2ffd640 1 -- 172.21.15.33:0/1698290490 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22037 (secure 0 0 0) 0x7fd2c40bc260 con 0x7fd2cc1017c0 2024-01-25T16:05:47.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:47.418 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":85,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:05:46.107586+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 msgr2=0x7fd2b80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80058c0 tx=0x7fd2c8005850 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 msgr2=0x7fd2cc19df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4037040 tx=0x7fd2c4032810 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 shutdown_connections 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc1017c0 0x7fd2cc19df50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100e00 0x7fd2cc19b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 --2- 172.21.15.33:0/1698290490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc0ffc10 0x7fd2cc069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 >> 172.21.15.33:0/1698290490 conn(0x7fd2cc0fb180 msgr2=0x7fd2cc104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 shutdown_connections 2024-01-25T16:05:47.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.417+0000 7fd2d3b5a640 1 -- 172.21.15.33:0/1698290490 wait complete. 2024-01-25T16:05:47.431 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-25T16:05:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 -- 172.21.15.33:0/226858299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c102950 msgr2=0x7f802c1091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 --2- 172.21.15.33:0/226858299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c102950 0x7f802c1091e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8024009ca0 tx=0x7f802402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 -- 172.21.15.33:0/226858299 shutdown_connections 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 --2- 172.21.15.33:0/226858299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c102950 0x7f802c1091e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 --2- 172.21.15.33:0/226858299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c101f90 0x7f802c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 --2- 172.21.15.33:0/226858299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c0fefc0 0x7f802c0ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 -- 172.21.15.33:0/226858299 >> 172.21.15.33:0/226858299 conn(0x7f802c0fa910 msgr2=0x7f802c0fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 -- 172.21.15.33:0/226858299 shutdown_connections 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.533+0000 7f8032831640 1 -- 172.21.15.33:0/226858299 wait complete. 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 Processor -- start 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 -- start start 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c101f90 0x7f802c1971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c102950 0x7f802c19e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19e790 con 0x7f802c0fefc0 2024-01-25T16:05:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19e900 con 0x7f802c101f90 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f8032831640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c19eac0 con 0x7f802c102950 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f802bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f802bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54530/0 (socket says 172.21.15.33:54530) 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.534+0000 7f802bfff640 1 -- 172.21.15.33:0/613413676 learned_addr learned my addr 172.21.15.33:0/613413676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 -- 172.21.15.33:0/613413676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c102950 msgr2=0x7f802c19e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f8030dce640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c102950 0x7f802c19e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c102950 0x7f802c19e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802b7fe640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c101f90 0x7f802c1971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 -- 172.21.15.33:0/613413676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c101f90 msgr2=0x7f802c1971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c101f90 0x7f802c1971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 -- 172.21.15.33:0/613413676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024009900 con 0x7f802c0fefc0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f802bfff640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b450 tx=0x7f801c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c011890 con 0x7f802c0fefc0 2024-01-25T16:05:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f801c0103e0 con 0x7f802c0fefc0 2024-01-25T16:05:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00f5a0 con 0x7f802c0fefc0 2024-01-25T16:05:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c19eda0 con 0x7f802c0fefc0 2024-01-25T16:05:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.535+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c19f270 con 0x7f802c0fefc0 2024-01-25T16:05:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.536+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c108060 con 0x7f802c0fefc0 2024-01-25T16:05:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.537+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f801c010850 con 0x7f802c0fefc0 2024-01-25T16:05:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.538+0000 7f80297fa640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 0x7f80180a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.538+0000 7f802b7fe640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 0x7f80180a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.538+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f801c0ccd10 con 0x7f802c0fefc0 2024-01-25T16:05:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.538+0000 7f802b7fe640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 0x7f80180a37a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f802c198180 tx=0x7f8014005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.541+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f801c096240 con 0x7f802c0fefc0 2024-01-25T16:05:47.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.763+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f802c10c640 con 0x7f802c0fefc0 2024-01-25T16:05:47.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.782+0000 7f80297fa640 1 -- 172.21.15.33:0/613413676 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f801c099860 con 0x7f802c0fefc0 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 msgr2=0x7f80180a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 0x7f80180a37a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f802c198180 tx=0x7f8014005cc0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 msgr2=0x7f802c196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f801c00b450 tx=0x7f801c00b910 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 shutdown_connections 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f802c102950 0x7f802c19e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f802c101f90 0x7f802c1971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80180a12e0 0x7f80180a37a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 --2- 172.21.15.33:0/613413676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f802c0fefc0 0x7f802c196c80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 >> 172.21.15.33:0/613413676 conn(0x7f802c0fa910 msgr2=0x7f802c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 shutdown_connections 2024-01-25T16:05:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.784+0000 7f8032831640 1 -- 172.21.15.33:0/613413676 wait complete. 2024-01-25T16:05:47.799 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-01-25T16:05:47.799 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-01-25T16:05:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 -- 172.21.15.33:0/1570430159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 msgr2=0x7faaa0100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 --2- 172.21.15.33:0/1570430159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa0100d00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faa9c009900 tx=0x7faa9c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 -- 172.21.15.33:0/1570430159 shutdown_connections 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 --2- 172.21.15.33:0/1570430159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa0100d00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 --2- 172.21.15.33:0/1570430159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 0x7faaa0100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 --2- 172.21.15.33:0/1570430159 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 0x7faaa00ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.901+0000 7faaa7db0640 1 -- 172.21.15.33:0/1570430159 >> 172.21.15.33:0/1570430159 conn(0x7faaa00fb1a0 msgr2=0x7faaa00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- 172.21.15.33:0/1570430159 shutdown_connections 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- 172.21.15.33:0/1570430159 wait complete. 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 Processor -- start 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- start start 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 0x7faaa0198e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 0x7faaa0199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa01a08e0 con 0x7faaa0100880 2024-01-25T16:05:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa01a0a50 con 0x7faaa00ffee0 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa7db0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa01a0c10 con 0x7faaa00ff580 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa634d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa634d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54544/0 (socket says 172.21.15.33:54544) 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa634d640 1 -- 172.21.15.33:0/3693540961 learned_addr learned my addr 172.21.15.33:0/3693540961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.902+0000 7faaa534b640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 0x7faaa0199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa5b4c640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 0x7faaa0198e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 -- 172.21.15.33:0/3693540961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 msgr2=0x7faaa0198e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 0x7faaa0198e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 -- 172.21.15.33:0/3693540961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 msgr2=0x7faaa0199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 0x7faaa0199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 -- 172.21.15.33:0/3693540961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009560 con 0x7faaa0100880 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa634d640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa9c02c7e0 tx=0x7faa9c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c005000 con 0x7faaa0100880 2024-01-25T16:05:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa9c005620 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c03e450 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa01a0e90 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.903+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa01a1310 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.904+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa010a2d0 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.905+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7faa9c005370 con 0x7faaa0100880 2024-01-25T16:05:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.906+0000 7faa96ffd640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 0x7faa8c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.906+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa9c0efb80 con 0x7faaa0100880 2024-01-25T16:05:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.906+0000 7faaa5b4c640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 0x7faa8c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.906+0000 7faaa5b4c640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 0x7faa8c0a37a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa98005c40 tx=0x7faa9800a390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:47.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:47.909+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7faa9c0b9270 con 0x7faaa0100880 2024-01-25T16:05:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-25T16:05:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: osdmap e85: 12 total, 12 up, 12 in 2024-01-25T16:05:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-25T16:05:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-25T16:05:48.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-25T16:05:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1221094918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1698290490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-25T16:05:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.127+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7faaa002ca70 con 0x7faaa0100880 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: osdmap e85: 12 total, 12 up, 12 in 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1221094918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1698290490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-25T16:05:48.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-01-25T16:05:48.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: osdmap e85: 12 total, 12 up, 12 in 2024-01-25T16:05:48.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4079460077' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-01-25T16:05:48.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-01-25T16:05:48.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-01-25T16:05:48.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1221094918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1698290490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:48.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-01-25T16:05:48.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.794+0000 7faa96ffd640 1 -- 172.21.15.33:0/3693540961 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7faa9c043020 con 0x7faaa0100880 2024-01-25T16:05:48.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 msgr2=0x7faa8c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 0x7faa8c0a37a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faa98005c40 tx=0x7faa9800a390 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 msgr2=0x7faaa01a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7faa9c02c7e0 tx=0x7faa9c004270 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 shutdown_connections 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7faa8c0a12e0 0x7faa8c0a37a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaa0100880 0x7faaa01a03a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faaa00ffee0 0x7faaa0199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 --2- 172.21.15.33:0/3693540961 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7faaa00ff580 0x7faaa0198e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 >> 172.21.15.33:0/3693540961 conn(0x7faaa00fb1a0 msgr2=0x7faaa00fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 shutdown_connections 2024-01-25T16:05:48.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.797+0000 7faaa7db0640 1 -- 172.21.15.33:0/3693540961 wait complete. 2024-01-25T16:05:48.816 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-01-25T16:05:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 -- 172.21.15.33:0/1167413573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc100d10 msgr2=0x7fd2cc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 --2- 172.21.15.33:0/1167413573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc100d10 0x7fd2cc101130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009900 tx=0x7fd2c802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 -- 172.21.15.33:0/1167413573 shutdown_connections 2024-01-25T16:05:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 --2- 172.21.15.33:0/1167413573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc1028c0 0x7fd2cc109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 --2- 172.21.15.33:0/1167413573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc101f00 0x7fd2cc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 --2- 172.21.15.33:0/1167413573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc100d10 0x7fd2cc101130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 -- 172.21.15.33:0/1167413573 >> 172.21.15.33:0/1167413573 conn(0x7fd2cc0fc280 msgr2=0x7fd2cc0fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 -- 172.21.15.33:0/1167413573 shutdown_connections 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.924+0000 7fd2d3968640 1 -- 172.21.15.33:0/1167413573 wait complete. 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 Processor -- start 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 -- start start 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100d10 0x7fd2cc196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc1028c0 0x7fd2cc19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19e960 con 0x7fd2cc101f00 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19ead0 con 0x7fd2cc1028c0 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d3968640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19ec90 con 0x7fd2cc100d10 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:48.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50638/0 (socket says 172.21.15.33:50638) 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 -- 172.21.15.33:0/345899584 learned_addr learned my addr 172.21.15.33:0/345899584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d1704640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100d10 0x7fd2cc196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d1f05640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc1028c0 0x7fd2cc19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 -- 172.21.15.33:0/345899584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100d10 msgr2=0x7fd2cc196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100d10 0x7fd2cc196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 -- 172.21.15.33:0/345899584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc1028c0 msgr2=0x7fd2cc19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc1028c0 0x7fd2cc19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.925+0000 7fd2d0f03640 1 -- 172.21.15.33:0/345899584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8009560 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2d0f03640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b4a0 tx=0x7fd2bc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc010610 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2bc010c30 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc19ef70 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc017dd0 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.926+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc19f470 con 0x7fd2cc101f00 2024-01-25T16:05:48.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.927+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc1080f0 con 0x7fd2cc101f00 2024-01-25T16:05:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.929+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd2bc0173f0 con 0x7fd2cc101f00 2024-01-25T16:05:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.929+0000 7fd2c27fc640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.929+0000 7fd2d1704640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.929+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd2bc0cc680 con 0x7fd2cc101f00 2024-01-25T16:05:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.930+0000 7fd2d1704640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:48.933+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd2bc095bb0 con 0x7fd2cc101f00 2024-01-25T16:05:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-25T16:05:49.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:active 2024-01-25T16:05:49.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:49.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: osdmap e86: 12 total, 12 up, 12 in 2024-01-25T16:05:49.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: pgmap v286: 97 pgs: 27 creating+peering, 39 unknown, 31 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:49.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:48 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-25T16:05:49.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-25T16:05:49.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:active 2024-01-25T16:05:49.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:49.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: osdmap e86: 12 total, 12 up, 12 in 2024-01-25T16:05:49.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: pgmap v286: 97 pgs: 27 creating+peering, 39 unknown, 31 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:49.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:48 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-25T16:05:49.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.151+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fd2cc10c8a0 con 0x7fd2cc101f00 2024-01-25T16:05:49.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/613413676' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-01-25T16:05:49.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] up:active 2024-01-25T16:05:49.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:49.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: osdmap e86: 12 total, 12 up, 12 in 2024-01-25T16:05:49.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: pgmap v286: 97 pgs: 27 creating+peering, 39 unknown, 31 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:05:49.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:48 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-01-25T16:05:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.800+0000 7fd2c27fc640 1 -- 172.21.15.33:0/345899584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7fd2bc0991d0 con 0x7fd2cc101f00 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 msgr2=0x7fd2b80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8000c00 tx=0x7fd2c802c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 msgr2=0x7fd2cc197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00b4a0 tx=0x7fd2bc00b960 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 shutdown_connections 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2cc1028c0 0x7fd2cc19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2b80a12e0 0x7fd2b80a37a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2cc101f00 0x7fd2cc197400 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 --2- 172.21.15.33:0/345899584 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2cc100d10 0x7fd2cc196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 >> 172.21.15.33:0/345899584 conn(0x7fd2cc0fc280 msgr2=0x7fd2cc1052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 shutdown_connections 2024-01-25T16:05:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.803+0000 7fd2d3968640 1 -- 172.21.15.33:0/345899584 wait complete. 2024-01-25T16:05:49.822 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/3135863850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 msgr2=0x7f2f18101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/3135863850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18101260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009900 tx=0x7f2f0002bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/3135863850 shutdown_connections 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/3135863850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f181017a0 0x7f2f18108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/3135863850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18101260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/3135863850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f180ffbf0 0x7f2f18100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.930+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/3135863850 >> 172.21.15.33:0/3135863850 conn(0x7f2f180fb180 msgr2=0x7f2f180fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.931+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/3135863850 shutdown_connections 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.931+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/3135863850 wait complete. 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.931+0000 7f2f1cbf1640 1 Processor -- start 2024-01-25T16:05:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.931+0000 7f2f1cbf1640 1 -- start start 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 0x7f2f18196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f181017a0 0x7f2f1819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1819e8c0 con 0x7f2f18100de0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1819ea30 con 0x7f2f181017a0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1819ebf0 con 0x7f2f180ffbf0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50644/0 (socket says 172.21.15.33:50644) 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 -- 172.21.15.33:0/1538856850 learned_addr learned my addr 172.21.15.33:0/1538856850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f16d9d640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f181017a0 0x7f2f1819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1659c640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 0x7f2f18196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 -- 172.21.15.33:0/1538856850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 msgr2=0x7f2f18196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 0x7f2f18196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 -- 172.21.15.33:0/1538856850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f181017a0 msgr2=0x7f2f1819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f181017a0 0x7f2f1819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 -- 172.21.15.33:0/1538856850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f00009560 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1659c640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 0x7f2f18196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0dd9b640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f0002c7e0 tx=0x7f2f00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00005000 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1819ee70 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f00005620 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0003e450 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.932+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1819f2f0 con 0x7f2f18100de0 2024-01-25T16:05:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.933+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1810b490 con 0x7f2f18100de0 2024-01-25T16:05:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.934+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2f000036f0 con 0x7f2f18100de0 2024-01-25T16:05:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.935+0000 7f2f0ffff640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 0x7f2f040a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.935+0000 7f2f1659c640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 0x7f2f040a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.935+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2f000efcb0 con 0x7f2f18100de0 2024-01-25T16:05:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.935+0000 7f2f1659c640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 0x7f2f040a37a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005d00 tx=0x7f2f08005c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:49.939+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2f000b91e0 con 0x7f2f18100de0 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: Cluster is now healthy 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-25T16:05:50.064 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:50.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-25T16:05:50.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-25T16:05:50.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:49 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: Cluster is now healthy 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-25T16:05:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-25T16:05:50.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:49 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-25T16:05:50.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:50.160+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f2f1819f480 con 0x7f2f18100de0 2024-01-25T16:05:50.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-01-25T16:05:50.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: Cluster is now healthy 2024-01-25T16:05:50.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3693540961' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-01-25T16:05:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:49 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-01-25T16:05:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.058+0000 7f2f0ffff640 1 -- 172.21.15.33:0/1538856850 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v21) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2f00043020 con 0x7f2f18100de0 2024-01-25T16:05:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 msgr2=0x7f2f040a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 0x7f2f040a37a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f08005d00 tx=0x7f2f08005c90 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 msgr2=0x7f2f18197310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2f0002c7e0 tx=0x7f2f00004270 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 shutdown_connections 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f181017a0 0x7f2f1819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2f040a12e0 0x7f2f040a37a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f18100de0 0x7f2f18197310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 --2- 172.21.15.33:0/1538856850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2f180ffbf0 0x7f2f18196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 >> 172.21.15.33:0/1538856850 conn(0x7f2f180fb180 msgr2=0x7f2f18067ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 shutdown_connections 2024-01-25T16:05:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.061+0000 7f2f1cbf1640 1 -- 172.21.15.33:0/1538856850 wait complete. 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:active 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: Cluster is now healthy 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:active 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: pgmap v287: 97 pgs: 24 creating+peering, 4 unknown, 69 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-01-25T16:05:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:50 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-25T16:05:51.076 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-01-25T16:05:51.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:active 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: Cluster is now healthy 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-01-25T16:05:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:active 2024-01-25T16:05:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: pgmap v287: 97 pgs: 24 creating+peering, 4 unknown, 69 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-01-25T16:05:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:50 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-25T16:05:51.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/345899584' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-01-25T16:05:51.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] up:active 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: Cluster is now healthy 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] up:active 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: pgmap v287: 97 pgs: 24 creating+peering, 4 unknown, 69 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-01-25T16:05:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:50 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-01-25T16:05:51.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- 172.21.15.33:0/3214095189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 msgr2=0x7f57e01064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 --2- 172.21.15.33:0/3214095189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01064b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009900 tx=0x7f57d002bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- 172.21.15.33:0/3214095189 shutdown_connections 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 --2- 172.21.15.33:0/3214095189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57e01069f0 0x7f57e0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 --2- 172.21.15.33:0/3214095189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01064b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 --2- 172.21.15.33:0/3214095189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0103b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- 172.21.15.33:0/3214095189 >> 172.21.15.33:0/3214095189 conn(0x7f57e00fb1a0 msgr2=0x7f57e00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- 172.21.15.33:0/3214095189 shutdown_connections 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- 172.21.15.33:0/3214095189 wait complete. 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 Processor -- start 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 -- start start 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.182+0000 7f57e779e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57e01069f0 0x7f57e019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e019e850 con 0x7f57e01040c0 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e019e9c0 con 0x7f57e01069f0 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e019eb80 con 0x7f57e0101790 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50668/0 (socket says 172.21.15.33:50668) 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 -- 172.21.15.33:0/1107610611 learned_addr learned my addr 172.21.15.33:0/1107610611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 -- 172.21.15.33:0/1107610611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 msgr2=0x7f57e0196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e553a640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 -- 172.21.15.33:0/1107610611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57e01069f0 msgr2=0x7f57e019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57e01069f0 0x7f57e019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 -- 172.21.15.33:0/1107610611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0009560 con 0x7f57e01040c0 2024-01-25T16:05:51.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e553a640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e4d39640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009230 tx=0x7f57d00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d00044c0 con 0x7f57e01040c0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57d00054c0 con 0x7f57e01040c0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e019ee00 con 0x7f57e01040c0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d003e450 con 0x7f57e01040c0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.183+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e019f300 con 0x7f57e01040c0 2024-01-25T16:05:51.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.184+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e00fcc90 con 0x7f57e01040c0 2024-01-25T16:05:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.185+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f57d00036f0 con 0x7f57e01040c0 2024-01-25T16:05:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.186+0000 7f57d67fc640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 0x7f57cc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.186+0000 7f57e553a640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 0x7f57cc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.186+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f57d00ef5e0 con 0x7f57e01040c0 2024-01-25T16:05:51.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.186+0000 7f57e553a640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 0x7f57cc0a37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006fd0 tx=0x7f57dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.190+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f57d00b8b10 con 0x7f57e01040c0 2024-01-25T16:05:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.397+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f57e002cca0 con 0x7f57cc0a12e0 2024-01-25T16:05:51.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.421+0000 7f57d67fc640 1 -- 172.21.15.33:0/1107610611 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f57e002cca0 con 0x7f57cc0a12e0 2024-01-25T16:05:51.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 msgr2=0x7f57cc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 0x7f57cc0a37a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006fd0 tx=0x7f57dc008040 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 msgr2=0x7f57e01972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009230 tx=0x7f57d00040c0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 shutdown_connections 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57e01069f0 0x7f57e019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f57cc0a12e0 0x7f57cc0a37a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57e01040c0 0x7f57e01972a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 --2- 172.21.15.33:0/1107610611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f57e0101790 0x7f57e0196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 >> 172.21.15.33:0/1107610611 conn(0x7f57e00fb1a0 msgr2=0x7f57e01021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 shutdown_connections 2024-01-25T16:05:51.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.424+0000 7f57e779e640 1 -- 172.21.15.33:0/1107610611 wait complete. 2024-01-25T16:05:51.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3847906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf80ffc00 msgr2=0x7fadf8100020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3847906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf80ffc00 0x7fadf8100020 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fade8009900 tx=0x7fade802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3847906900 shutdown_connections 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3847906900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf81017b0 0x7fadf8108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3847906900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf8100df0 0x7fadf8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3847906900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf80ffc00 0x7fadf8100020 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3847906900 >> 172.21.15.33:0/3847906900 conn(0x7fadf80fb190 msgr2=0x7fadf80fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3847906900 shutdown_connections 2024-01-25T16:05:51.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3847906900 wait complete. 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 Processor -- start 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.543+0000 7fadfdb6f640 1 -- start start 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf80ffc00 0x7fadf8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf8100df0 0x7fadf81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf819e850 con 0x7fadf81017b0 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf819e9c0 con 0x7fadf8100df0 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf819eb80 con 0x7fadf80ffc00 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50720/0 (socket says 172.21.15.33:50720) 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 -- 172.21.15.33:0/3997124950 learned_addr learned my addr 172.21.15.33:0/3997124950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf77fe640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf80ffc00 0x7fadf8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf6ffd640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf8100df0 0x7fadf81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 -- 172.21.15.33:0/3997124950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf80ffc00 msgr2=0x7fadf8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf80ffc00 0x7fadf8196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 -- 172.21.15.33:0/3997124950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf8100df0 msgr2=0x7fadf81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf8100df0 0x7fadf81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 -- 172.21.15.33:0/3997124950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8009560 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf7fff640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b970 tx=0x7fadf000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000cb40 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadf000cca0 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf819ee60 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007770 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.544+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf819f310 con 0x7fadf81017b0 2024-01-25T16:05:51.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.545+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf810b510 con 0x7fadf81017b0 2024-01-25T16:05:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.546+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fadf0014440 con 0x7fadf81017b0 2024-01-25T16:05:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.547+0000 7fadf4ff9640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 0x7fade00a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.547+0000 7fadf77fe640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 0x7fade00a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.547+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadf00cc7d0 con 0x7fadf81017b0 2024-01-25T16:05:51.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.547+0000 7fadf77fe640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 0x7fade00a37a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fade8000c00 tx=0x7fade802c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.550+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fadf0095db0 con 0x7fadf81017b0 2024-01-25T16:05:51.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.758+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fadf80fbf70 con 0x7fade00a12e0 2024-01-25T16:05:51.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.780+0000 7fadf4ff9640 1 -- 172.21.15.33:0/3997124950 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fadf80fbf70 con 0x7fade00a12e0 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 msgr2=0x7fade00a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 0x7fade00a37a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fade8000c00 tx=0x7fade802c5d0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 msgr2=0x7fadf819e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fadf000b970 tx=0x7fadf000be30 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 shutdown_connections 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fade00a12e0 0x7fade00a37a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadf81017b0 0x7fadf819e310 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf8100df0 0x7fadf81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 --2- 172.21.15.33:0/3997124950 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fadf80ffc00 0x7fadf8196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.782+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 >> 172.21.15.33:0/3997124950 conn(0x7fadf80fb190 msgr2=0x7fadf80fb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.783+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 shutdown_connections 2024-01-25T16:05:51.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.783+0000 7fadfdb6f640 1 -- 172.21.15.33:0/3997124950 wait complete. 2024-01-25T16:05:51.795 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:05:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:05:51.402+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:05:51.795 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:05:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:05:51.402+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:05:51.795 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:05:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:05:51.402+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:05:51.795 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:05:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:05:51.402+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:05:51.795 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:05:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:05:51.402+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:05:51.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-01-25T16:05:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 -- 172.21.15.33:0/1867672397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 msgr2=0x7fc564108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1867672397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc564108030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc55c009ca0 tx=0x7fc55c02c100 comp rx=0 tx=0).stop 2024-01-25T16:05:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 -- 172.21.15.33:0/1867672397 shutdown_connections 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1867672397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc564108030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1867672397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 0x7fc564101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1867672397 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 0x7fc564100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.900+0000 7fc56b96e640 1 -- 172.21.15.33:0/1867672397 >> 172.21.15.33:0/1867672397 conn(0x7fc5640fb180 msgr2=0x7fc5640fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- 172.21.15.33:0/1867672397 shutdown_connections 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- 172.21.15.33:0/1867672397 wait complete. 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 Processor -- start 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- start start 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 0x7fc564196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 0x7fc564197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56419e640 con 0x7fc5641017a0 2024-01-25T16:05:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56419e7b0 con 0x7fc564100de0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56b96e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc56419e970 con 0x7fc5640ffbf0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc569f0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc569f0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50748/0 (socket says 172.21.15.33:50748) 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc569f0b640 1 -- 172.21.15.33:0/1527892745 learned_addr learned my addr 172.21.15.33:0/1527892745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.901+0000 7fc56970a640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 0x7fc564196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc568f09640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 0x7fc564197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 -- 172.21.15.33:0/1527892745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 msgr2=0x7fc564196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 0x7fc564196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 -- 172.21.15.33:0/1527892745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 msgr2=0x7fc564197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 0x7fc564197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 -- 172.21.15.33:0/1527892745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c009900 con 0x7fc5641017a0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc569f0b640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc55c037040 tx=0x7fc55c032ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c003970 con 0x7fc5641017a0 2024-01-25T16:05:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc55c004040 con 0x7fc5641017a0 2024-01-25T16:05:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56419ebf0 con 0x7fc5641017a0 2024-01-25T16:05:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55c040690 con 0x7fc5641017a0 2024-01-25T16:05:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.902+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56419f0f0 con 0x7fc5641017a0 2024-01-25T16:05:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.903+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc564106eb0 con 0x7fc5641017a0 2024-01-25T16:05:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.904+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fc55c00a040 con 0x7fc5641017a0 2024-01-25T16:05:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.904+0000 7fc55a7fc640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 0x7fc5500a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.905+0000 7fc56970a640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 0x7fc5500a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.905+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc55c0f03d0 con 0x7fc5641017a0 2024-01-25T16:05:51.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.905+0000 7fc56970a640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 0x7fc5500a3820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc564100c40 tx=0x7fc560005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:51.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:51.908+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc55c0b9aa0 con 0x7fc5641017a0 2024-01-25T16:05:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.132+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fc56402cbe0 con 0x7fc5500a1360 2024-01-25T16:05:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.136+0000 7fc55a7fc640 1 -- 172.21.15.33:0/1527892745 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fc56402cbe0 con 0x7fc5500a1360 2024-01-25T16:05:52.139 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/00fe401d-85e6-4b4b-9352-804377517a18 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 msgr2=0x7fc5500a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 0x7fc5500a3820 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc564100c40 tx=0x7fc560005f90 comp rx=0 tx=0).stop 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 msgr2=0x7fc56419e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc55c037040 tx=0x7fc55c032ae0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 shutdown_connections 2024-01-25T16:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc5500a1360 0x7fc5500a3820 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5641017a0 0x7fc56419e100 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc564100de0 0x7fc564197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 --2- 172.21.15.33:0/1527892745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5640ffbf0 0x7fc564196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 >> 172.21.15.33:0/1527892745 conn(0x7fc5640fb180 msgr2=0x7fc564104080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 shutdown_connections 2024-01-25T16:05:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.138+0000 7fc56b96e640 1 -- 172.21.15.33:0/1527892745 wait complete. 2024-01-25T16:05:52.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-01-25T16:05:52.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 -- 172.21.15.33:0/3382372090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c1017a0 msgr2=0x7f010c108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3382372090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c1017a0 0x7f010c108030 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0108009ca0 tx=0x7f010802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 -- 172.21.15.33:0/3382372090 shutdown_connections 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3382372090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c1017a0 0x7f010c108030 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3382372090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c100de0 0x7f010c101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3382372090 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c0ffbf0 0x7f010c100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 -- 172.21.15.33:0/3382372090 >> 172.21.15.33:0/3382372090 conn(0x7f010c0fb180 msgr2=0x7f010c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 -- 172.21.15.33:0/3382372090 shutdown_connections 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.256+0000 7f0113f5a640 1 -- 172.21.15.33:0/3382372090 wait complete. 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 Processor -- start 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 -- start start 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c0ffbf0 0x7f010c196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c1017a0 0x7f010c19e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19e8f0 con 0x7f010c100de0 2024-01-25T16:05:52.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19ea60 con 0x7f010c0ffbf0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0113f5a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19ec20 con 0x7f010c1017a0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f01114f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f01114f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50768/0 (socket says 172.21.15.33:50768) 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f01114f5640 1 -- 172.21.15.33:0/3920267125 learned_addr learned my addr 172.21.15.33:0/3920267125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f0111cf6640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c0ffbf0 0x7f010c196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.257+0000 7f01124f7640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c1017a0 0x7f010c19e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 -- 172.21.15.33:0/3920267125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c1017a0 msgr2=0x7f010c19e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c1017a0 0x7f010c19e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 -- 172.21.15.33:0/3920267125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c0ffbf0 msgr2=0x7f010c196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c0ffbf0 0x7f010c196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 -- 172.21.15.33:0/3920267125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0108009900 con 0x7f010c100de0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f01114f5640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b4a0 tx=0x7f00fc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc011830 con 0x7f010c100de0 2024-01-25T16:05:52.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00fc011e50 con 0x7f010c100de0 2024-01-25T16:05:52.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c19ef00 con 0x7f010c100de0 2024-01-25T16:05:52.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00fc00f2e0 con 0x7f010c100de0 2024-01-25T16:05:52.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.258+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c19f400 con 0x7f010c100de0 2024-01-25T16:05:52.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.259+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c106fe0 con 0x7f010c100de0 2024-01-25T16:05:52.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.260+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f00fc0105a0 con 0x7f010c100de0 2024-01-25T16:05:52.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.260+0000 7f0102ffd640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 0x7f00f80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.261+0000 7f0111cf6640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 0x7f00f80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.261+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f00fc0cca80 con 0x7f010c100de0 2024-01-25T16:05:52.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.261+0000 7f0111cf6640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 0x7f00f80a37a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f010c100c40 tx=0x7f0104009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.264+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f00fc096050 con 0x7f010c100de0 2024-01-25T16:05:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:52 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-25T16:05:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:52 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:52.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:52 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-25T16:05:52.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:52 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:52 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1538856850' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-01-25T16:05:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:52 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:05:52.419 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: 2024-01-25T16:05:52.098+0000 7fe8b1363700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-01-25T16:05:52.419 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: 2024-01-25T16:05:52.098+0000 7fe8b1363700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-01-25T16:05:52.419 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:05:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: 2024-01-25T16:05:52.098+0000 7fe8b1363700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-01-25T16:05:52.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.471+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f010c02cbe0 con 0x7f00f80a12e0 2024-01-25T16:05:52.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.494+0000 7f0102ffd640 1 -- 172.21.15.33:0/3920267125 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f010c02cbe0 con 0x7f00f80a12e0 2024-01-25T16:05:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.496+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 msgr2=0x7f00f80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.496+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 0x7f00f80a37a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f010c100c40 tx=0x7f0104009110 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.496+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 msgr2=0x7f010c197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.496+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b4a0 tx=0x7f00fc00b8a0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.496+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 shutdown_connections 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f010c1017a0 0x7f010c19e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f00f80a12e0 0x7f00f80a37a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f010c100de0 0x7f010c197340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 --2- 172.21.15.33:0/3920267125 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f010c0ffbf0 0x7f010c196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 >> 172.21.15.33:0/3920267125 conn(0x7f010c0fb180 msgr2=0x7f010c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 shutdown_connections 2024-01-25T16:05:52.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.497+0000 7f0113f5a640 1 -- 172.21.15.33:0/3920267125 wait complete. 2024-01-25T16:05:52.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-01-25T16:05:52.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 -- 172.21.15.33:0/4088614896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad48100df0 msgr2=0x7fad48101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/4088614896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad48100df0 0x7fad48101270 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009900 tx=0x7fad3c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:52.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 -- 172.21.15.33:0/4088614896 shutdown_connections 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/4088614896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad481017b0 0x7fad48108040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/4088614896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad48100df0 0x7fad48101270 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/4088614896 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad480ffc00 0x7fad48100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 -- 172.21.15.33:0/4088614896 >> 172.21.15.33:0/4088614896 conn(0x7fad480fb190 msgr2=0x7fad480fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 -- 172.21.15.33:0/4088614896 shutdown_connections 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.614+0000 7fad4e9f8640 1 -- 172.21.15.33:0/4088614896 wait complete. 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 Processor -- start 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 -- start start 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad480ffc00 0x7fad48196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48100df0 0x7fad48197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819e840 con 0x7fad481017b0 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819e9b0 con 0x7fad480ffc00 2024-01-25T16:05:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4e9f8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4819eb70 con 0x7fad48100df0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4cf95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4cf95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50772/0 (socket says 172.21.15.33:50772) 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.615+0000 7fad4cf95640 1 -- 172.21.15.33:0/3723322438 learned_addr learned my addr 172.21.15.33:0/3723322438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad477fe640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48100df0 0x7fad48197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 -- 172.21.15.33:0/3723322438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48100df0 msgr2=0x7fad48197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48100df0 0x7fad48197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 -- 172.21.15.33:0/3723322438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad480ffc00 msgr2=0x7fad48196d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad480ffc00 0x7fad48196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 -- 172.21.15.33:0/3723322438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c009560 con 0x7fad481017b0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4cf95640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad4000b970 tx=0x7fad4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4000cb40 con 0x7fad481017b0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad4819ee50 con 0x7fad481017b0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad4000cca0 con 0x7fad481017b0 2024-01-25T16:05:52.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad40007770 con 0x7fad481017b0 2024-01-25T16:05:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.616+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad4819f300 con 0x7fad481017b0 2024-01-25T16:05:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.617+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4810b540 con 0x7fad481017b0 2024-01-25T16:05:52.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.618+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fad40014440 con 0x7fad481017b0 2024-01-25T16:05:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.618+0000 7fad457fa640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 0x7fad300a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.619+0000 7fad47fff640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 0x7fad300a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.619+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad400cc790 con 0x7fad481017b0 2024-01-25T16:05:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.619+0000 7fad47fff640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 0x7fad300a37a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad48100c50 tx=0x7fad38005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.623+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fad40095d60 con 0x7fad481017b0 2024-01-25T16:05:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.832+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fad480fd3c0 con 0x7fad300a12e0 2024-01-25T16:05:52.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.836+0000 7fad457fa640 1 -- 172.21.15.33:0/3723322438 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fad480fd3c0 con 0x7fad300a12e0 2024-01-25T16:05:52.839 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/960ef3eb-54d7-4234-b24f-2801405b107f 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 msgr2=0x7fad300a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 0x7fad300a37a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fad48100c50 tx=0x7fad38005c10 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 msgr2=0x7fad4819e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad4000b970 tx=0x7fad4000be30 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 shutdown_connections 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fad300a12e0 0x7fad300a37a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad481017b0 0x7fad4819e300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fad48100df0 0x7fad48197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 --2- 172.21.15.33:0/3723322438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad480ffc00 0x7fad48196d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 >> 172.21.15.33:0/3723322438 conn(0x7fad480fb190 msgr2=0x7fad480fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 shutdown_connections 2024-01-25T16:05:52.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.838+0000 7fad4e9f8640 1 -- 172.21.15.33:0/3723322438 wait complete. 2024-01-25T16:05:52.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 -- 172.21.15.33:0/2223715726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46941017c0 msgr2=0x7f4694108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 --2- 172.21.15.33:0/2223715726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46941017c0 0x7f4694108050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f468c009ca0 tx=0x7f468c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 -- 172.21.15.33:0/2223715726 shutdown_connections 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 --2- 172.21.15.33:0/2223715726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46941017c0 0x7f4694108050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 --2- 172.21.15.33:0/2223715726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4694100e00 0x7f4694101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 --2- 172.21.15.33:0/2223715726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 0x7f4694100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 -- 172.21.15.33:0/2223715726 >> 172.21.15.33:0/2223715726 conn(0x7f46940fb180 msgr2=0x7f46940fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:52.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 -- 172.21.15.33:0/2223715726 shutdown_connections 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 -- 172.21.15.33:0/2223715726 wait complete. 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.951+0000 7f469a02f640 1 Processor -- start 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 -- start start 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 0x7f4694198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46941017c0 0x7f46941a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46941a08f0 con 0x7f4694100e00 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46941a0a60 con 0x7f46941017c0 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f469a02f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46941a0c20 con 0x7f46940ffc10 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50786/0 (socket says 172.21.15.33:50786) 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 -- 172.21.15.33:0/2272765719 learned_addr learned my addr 172.21.15.33:0/2272765719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f46937fe640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 0x7f4694198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4693fff640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46941017c0 0x7f46941a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 -- 172.21.15.33:0/2272765719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 msgr2=0x7f4694198e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 0x7f4694198e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 -- 172.21.15.33:0/2272765719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46941017c0 msgr2=0x7f46941a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46941017c0 0x7f46941a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 -- 172.21.15.33:0/2272765719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c009900 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4692ffd640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f468800c920 tx=0x7f468800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.952+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688007c70 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.953+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46941a0f00 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.953+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4688007dd0 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.953+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880187c0 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.953+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46941a13d0 con 0x7f4694100e00 2024-01-25T16:05:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.953+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469410b4d0 con 0x7f4694100e00 2024-01-25T16:05:52.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.955+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f468800f450 con 0x7f4694100e00 2024-01-25T16:05:52.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.955+0000 7f4690ff9640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 0x7f467c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:52.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.955+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f46880ccda0 con 0x7f4694100e00 2024-01-25T16:05:52.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.955+0000 7f46937fe640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 0x7f467c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:52.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.956+0000 7f46937fe640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 0x7f467c0a37a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46840058c0 tx=0x7f468400a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:52.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:52.959+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4688096400 con 0x7f4694100e00 2024-01-25T16:05:53.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.077+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f468800f700 con 0x7f4694100e00 2024-01-25T16:05:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.180+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f469410b690 con 0x7f4694100e00 2024-01-25T16:05:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.181+0000 7f4690ff9640 1 -- 172.21.15.33:0/2272765719 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f469410b690 con 0x7f4694100e00 2024-01-25T16:05:53.183 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:53.183 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:05:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 msgr2=0x7f467c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 0x7f467c0a37a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f46840058c0 tx=0x7f468400a2d0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 msgr2=0x7f4694199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f468800c920 tx=0x7f468800cde0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 shutdown_connections 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46941017c0 0x7f46941a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f467c0a12e0 0x7f467c0a37a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4694100e00 0x7f4694199340 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 --2- 172.21.15.33:0/2272765719 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46940ffc10 0x7f4694198e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 >> 172.21.15.33:0/2272765719 conn(0x7f46940fb180 msgr2=0x7f46940fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 shutdown_connections 2024-01-25T16:05:53.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.183+0000 7f469a02f640 1 -- 172.21.15.33:0/2272765719 wait complete. 2024-01-25T16:05:53.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:53 smithi033 ceph-mon[31417]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:53 smithi033 ceph-mon[31417]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.195 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:53 smithi033 ceph-mon[31417]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-01-25T16:05:53.196 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:53 smithi033 ceph-mon[31417]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 -- 172.21.15.33:0/3984607693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 msgr2=0x7f72a4101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 --2- 172.21.15.33:0/3984607693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4101280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7298009900 tx=0x7f729802bc30 comp rx=0 tx=0).stop 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 -- 172.21.15.33:0/3984607693 shutdown_connections 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 --2- 172.21.15.33:0/3984607693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a41017c0 0x7f72a4108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 --2- 172.21.15.33:0/3984607693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4101280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 --2- 172.21.15.33:0/3984607693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a40ffc10 0x7f72a4100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 -- 172.21.15.33:0/3984607693 >> 172.21.15.33:0/3984607693 conn(0x7f72a40fb180 msgr2=0x7f72a40fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 -- 172.21.15.33:0/3984607693 shutdown_connections 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.296+0000 7f72ac231640 1 -- 172.21.15.33:0/3984607693 wait complete. 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 Processor -- start 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 -- start start 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 0x7f72a4196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 0x7f72a419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419e920 con 0x7f72a4100e00 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419ea90 con 0x7f72a41017c0 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72ac231640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a419ec50 con 0x7f72a40ffc10 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50796/0 (socket says 172.21.15.33:50796) 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 -- 172.21.15.33:0/3056421313 learned_addr learned my addr 172.21.15.33:0/3056421313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72aa7ce640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 0x7f72a419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a9fcd640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 0x7f72a4196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 -- 172.21.15.33:0/3056421313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 msgr2=0x7f72a4196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 0x7f72a4196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 -- 172.21.15.33:0/3056421313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 msgr2=0x7f72a419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 0x7f72a419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a97cc640 1 -- 172.21.15.33:0/3056421313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298009560 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72a9fcd640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 0x7f72a4196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.297+0000 7f72aa7ce640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 0x7f72a419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72a97cc640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7298009230 tx=0x7f72980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980044e0 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a419eed0 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72980035f0 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729803e6b0 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.298+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a419f420 con 0x7f72a4100e00 2024-01-25T16:05:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.299+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a4107000 con 0x7f72a4100e00 2024-01-25T16:05:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.300+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f7298003960 con 0x7f72a4100e00 2024-01-25T16:05:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.300+0000 7f72a2ffd640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 0x7f728c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.300+0000 7f72a9fcd640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 0x7f728c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.300+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72980ef900 con 0x7f72a4100e00 2024-01-25T16:05:53.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.301+0000 7f72a9fcd640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 0x7f728c0a3830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.304+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f72980b8ee0 con 0x7f72a4100e00 2024-01-25T16:05:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:53 smithi152 ceph-mon[36015]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:53 smithi152 ceph-mon[36015]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:53 smithi152 ceph-mon[36015]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-01-25T16:05:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:53 smithi152 ceph-mon[36015]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:53 smithi076 ceph-mon[35642]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:53 smithi076 ceph-mon[35642]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:53 smithi076 ceph-mon[35642]: pgmap v288: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-01-25T16:05:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:53 smithi076 ceph-mon[35642]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:53.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.484+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a410c840 con 0x7f72a4100e00 2024-01-25T16:05:53.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.485+0000 7f72a2ffd640 1 -- 172.21.15.33:0/3056421313 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22024 (secure 0 0 0) 0x7f72a410c840 con 0x7f72a4100e00 2024-01-25T16:05:53.488 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:53.488 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":86,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:05:47.781903+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 msgr2=0x7f728c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 0x7f728c0a3830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f729c006fd0 tx=0x7f729c008040 comp rx=0 tx=0).stop 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 msgr2=0x7f72a4197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7298009230 tx=0x7f72980042f0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 shutdown_connections 2024-01-25T16:05:53.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72a41017c0 0x7f72a419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f728c0a1370 0x7f728c0a3830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72a4100e00 0x7f72a4197370 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 --2- 172.21.15.33:0/3056421313 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72a40ffc10 0x7f72a4196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 >> 172.21.15.33:0/3056421313 conn(0x7f72a40fb180 msgr2=0x7f72a41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 shutdown_connections 2024-01-25T16:05:53.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.487+0000 7f72ac231640 1 -- 172.21.15.33:0/3056421313 wait complete. 2024-01-25T16:05:53.501 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-25T16:05:53.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-25T16:05:53.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- 172.21.15.33:0/3618043750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98102960 msgr2=0x7f0f981091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 --2- 172.21.15.33:0/3618043750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98102960 0x7f0f981091f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f90009ca0 tx=0x7f0f9002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- 172.21.15.33:0/3618043750 shutdown_connections 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 --2- 172.21.15.33:0/3618043750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98102960 0x7f0f981091f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 --2- 172.21.15.33:0/3618043750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f98101fa0 0x7f0f98102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 --2- 172.21.15.33:0/3618043750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f980fefd0 0x7f0f980ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- 172.21.15.33:0/3618043750 >> 172.21.15.33:0/3618043750 conn(0x7f0f980fa920 msgr2=0x7f0f980fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- 172.21.15.33:0/3618043750 shutdown_connections 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- 172.21.15.33:0/3618043750 wait complete. 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 Processor -- start 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.601+0000 7f0f9e168640 1 -- start start 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f980fefd0 0x7f0f98196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f98102960 0x7f0f9819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819e8b0 con 0x7f0f98101fa0 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819ea20 con 0x7f0f98102960 2024-01-25T16:05:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9819ebe0 con 0x7f0f980fefd0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50818/0 (socket says 172.21.15.33:50818) 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 -- 172.21.15.33:0/1838249745 learned_addr learned my addr 172.21.15.33:0/1838249745 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f97fff640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f98102960 0x7f0f9819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f977fe640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f980fefd0 0x7f0f98196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 -- 172.21.15.33:0/1838249745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f980fefd0 msgr2=0x7f0f98196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f980fefd0 0x7f0f98196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 -- 172.21.15.33:0/1838249745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f98102960 msgr2=0x7f0f9819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f98102960 0x7f0f9819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 -- 172.21.15.33:0/1838249745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f90009900 con 0x7f0f98101fa0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f96ffd640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00b4a0 tx=0x7f0f8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c010610 con 0x7f0f98101fa0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9819eec0 con 0x7f0f98101fa0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8c010c30 con 0x7f0f98101fa0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c017dd0 con 0x7f0f98101fa0 2024-01-25T16:05:53.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.602+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9819f410 con 0x7f0f98101fa0 2024-01-25T16:05:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.603+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f981081f0 con 0x7f0f98101fa0 2024-01-25T16:05:53.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.604+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f0f8c0173f0 con 0x7f0f98101fa0 2024-01-25T16:05:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.605+0000 7f0f94ff9640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 0x7f0f800a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.605+0000 7f0f977fe640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 0x7f0f800a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.605+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0f8c0cc210 con 0x7f0f98101fa0 2024-01-25T16:05:53.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.605+0000 7f0f977fe640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 0x7f0f800a3830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f980691c0 tx=0x7f0f88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.609+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0f8c0956f0 con 0x7f0f98101fa0 2024-01-25T16:05:53.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.786+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f9802cbe0 con 0x7f0f800a1370 2024-01-25T16:05:53.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.797+0000 7f0f94ff9640 1 -- 172.21.15.33:0/1838249745 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336235 (secure 0 0 0) 0x7f0f9802cbe0 con 0x7f0f800a1370 2024-01-25T16:05:53.804 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 msgr2=0x7f0f800a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 0x7f0f800a3830 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0f980691c0 tx=0x7f0f88008040 comp rx=0 tx=0).stop 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 msgr2=0x7f0f98197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00b4a0 tx=0x7f0f8c00b960 comp rx=0 tx=0).stop 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 shutdown_connections 2024-01-25T16:05:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f98102960 0x7f0f9819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f800a1370 0x7f0f800a3830 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f98101fa0 0x7f0f98197350 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 --2- 172.21.15.33:0/1838249745 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f980fefd0 0x7f0f98196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.800+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 >> 172.21.15.33:0/1838249745 conn(0x7f0f980fa920 msgr2=0x7f0f981053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.801+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 shutdown_connections 2024-01-25T16:05:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.801+0000 7f0f9e168640 1 -- 172.21.15.33:0/1838249745 wait complete. 2024-01-25T16:05:53.814 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-01-25T16:05:52.099756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595174,"num_objects":47,"num_object_clones":0,"num_object_copies":141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":0,"num_read":158,"num_read_kb":136,"num_write":279,"num_write_kb":2966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":262,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366104,"kb_used_data":12744,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575720,"statfs":{"total":1151940427776,"available":1151565537280,"internally_reserved":0,"allocated":13049856,"data_stored":7077174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4806,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":98,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.345894"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809413+0000","last_change":"2024-01-25T16:05:46.796305+0000","last_active":"2024-01-25T16:05:48.809413+0000","last_peered":"2024-01-25T16:05:48.809413+0000","last_clean":"2024-01-25T16:05:48.809413+0000","last_became_active":"2024-01-25T16:05:46.795989+0000","last_became_peered":"2024-01-25T16:05:46.795989+0000","last_unstale":"2024-01-25T16:05:48.809413+0000","last_undegraded":"2024-01-25T16:05:48.809413+0000","last_fullsized":"2024-01-25T16:05:48.809413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:45.764391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.799735+0000","last_change":"2024-01-25T16:05:46.798335+0000","last_active":"2024-01-25T16:05:46.799735+0000","last_peered":"2024-01-25T16:05:46.799735+0000","last_clean":"2024-01-25T16:05:46.799735+0000","last_became_active":"2024-01-25T16:05:46.797743+0000","last_became_peered":"2024-01-25T16:05:46.797743+0000","last_unstale":"2024-01-25T16:05:46.799735+0000","last_undegraded":"2024-01-25T16:05:46.799735+0000","last_fullsized":"2024-01-25T16:05:46.799735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:57:02.878776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.804010+0000","last_change":"2024-01-25T16:05:46.803057+0000","last_active":"2024-01-25T16:05:46.804010+0000","last_peered":"2024-01-25T16:05:46.804010+0000","last_clean":"2024-01-25T16:05:46.804010+0000","last_became_active":"2024-01-25T16:05:46.802706+0000","last_became_peered":"2024-01-25T16:05:46.802706+0000","last_unstale":"2024-01-25T16:05:46.804010+0000","last_undegraded":"2024-01-25T16:05:46.804010+0000","last_fullsized":"2024-01-25T16:05:46.804010+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:46:36.681569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791942+0000","last_change":"2024-01-25T16:05:46.796172+0000","last_active":"2024-01-25T16:05:47.791942+0000","last_peered":"2024-01-25T16:05:47.791942+0000","last_clean":"2024-01-25T16:05:47.791942+0000","last_became_active":"2024-01-25T16:05:46.795848+0000","last_became_peered":"2024-01-25T16:05:46.795848+0000","last_unstale":"2024-01-25T16:05:47.791942+0000","last_undegraded":"2024-01-25T16:05:47.791942+0000","last_fullsized":"2024-01-25T16:05:47.791942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:41:07.553896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.795837+0000","last_change":"2024-01-25T16:05:46.794790+0000","last_active":"2024-01-25T16:05:46.795837+0000","last_peered":"2024-01-25T16:05:46.795837+0000","last_clean":"2024-01-25T16:05:46.795837+0000","last_became_active":"2024-01-25T16:05:46.794464+0000","last_became_peered":"2024-01-25T16:05:46.794464+0000","last_unstale":"2024-01-25T16:05:46.795837+0000","last_undegraded":"2024-01-25T16:05:46.795837+0000","last_fullsized":"2024-01-25T16:05:46.795837+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:08:01.843136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808136+0000","last_change":"2024-01-25T16:05:46.794263+0000","last_active":"2024-01-25T16:05:48.808136+0000","last_peered":"2024-01-25T16:05:48.808136+0000","last_clean":"2024-01-25T16:05:48.808136+0000","last_became_active":"2024-01-25T16:05:46.793598+0000","last_became_peered":"2024-01-25T16:05:46.793598+0000","last_unstale":"2024-01-25T16:05:48.808136+0000","last_undegraded":"2024-01-25T16:05:48.808136+0000","last_fullsized":"2024-01-25T16:05:48.808136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:47:25.071887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791874+0000","last_change":"2024-01-25T16:05:46.801432+0000","last_active":"2024-01-25T16:05:47.791874+0000","last_peered":"2024-01-25T16:05:47.791874+0000","last_clean":"2024-01-25T16:05:47.791874+0000","last_became_active":"2024-01-25T16:05:46.796673+0000","last_became_peered":"2024-01-25T16:05:46.796673+0000","last_unstale":"2024-01-25T16:05:47.791874+0000","last_undegraded":"2024-01-25T16:05:47.791874+0000","last_fullsized":"2024-01-25T16:05:47.791874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T04:02:37.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790607+0000","last_change":"2024-01-25T16:05:46.805885+0000","last_active":"2024-01-25T16:05:47.790607+0000","last_peered":"2024-01-25T16:05:47.790607+0000","last_clean":"2024-01-25T16:05:47.790607+0000","last_became_active":"2024-01-25T16:05:46.805359+0000","last_became_peered":"2024-01-25T16:05:46.805359+0000","last_unstale":"2024-01-25T16:05:47.790607+0000","last_undegraded":"2024-01-25T16:05:47.790607+0000","last_fullsized":"2024-01-25T16:05:47.790607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:44:54.641646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809008+0000","last_change":"2024-01-25T16:05:46.791944+0000","last_active":"2024-01-25T16:05:48.809008+0000","last_peered":"2024-01-25T16:05:48.809008+0000","last_clean":"2024-01-25T16:05:48.809008+0000","last_became_active":"2024-01-25T16:05:46.791637+0000","last_became_peered":"2024-01-25T16:05:46.791637+0000","last_unstale":"2024-01-25T16:05:48.809008+0000","last_undegraded":"2024-01-25T16:05:48.809008+0000","last_fullsized":"2024-01-25T16:05:48.809008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:54:20.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809017+0000","last_change":"2024-01-25T16:05:46.790961+0000","last_active":"2024-01-25T16:05:48.809017+0000","last_peered":"2024-01-25T16:05:48.809017+0000","last_clean":"2024-01-25T16:05:48.809017+0000","last_became_active":"2024-01-25T16:05:46.790664+0000","last_became_peered":"2024-01-25T16:05:46.790664+0000","last_unstale":"2024-01-25T16:05:48.809017+0000","last_undegraded":"2024-01-25T16:05:48.809017+0000","last_fullsized":"2024-01-25T16:05:48.809017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:24:57.615304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792526+0000","last_change":"2024-01-25T16:05:46.795205+0000","last_active":"2024-01-25T16:05:47.792526+0000","last_peered":"2024-01-25T16:05:47.792526+0000","last_clean":"2024-01-25T16:05:47.792526+0000","last_became_active":"2024-01-25T16:05:46.794894+0000","last_became_peered":"2024-01-25T16:05:46.794894+0000","last_unstale":"2024-01-25T16:05:47.792526+0000","last_undegraded":"2024-01-25T16:05:47.792526+0000","last_fullsized":"2024-01-25T16:05:47.792526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:57:37.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791897+0000","last_change":"2024-01-25T16:05:46.795473+0000","last_active":"2024-01-25T16:05:47.791897+0000","last_peered":"2024-01-25T16:05:47.791897+0000","last_clean":"2024-01-25T16:05:47.791897+0000","last_became_active":"2024-01-25T16:05:46.795155+0000","last_became_peered":"2024-01-25T16:05:46.795155+0000","last_unstale":"2024-01-25T16:05:47.791897+0000","last_undegraded":"2024-01-25T16:05:47.791897+0000","last_fullsized":"2024-01-25T16:05:47.791897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:42:25.273806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809704+0000","last_change":"2024-01-25T16:05:46.792550+0000","last_active":"2024-01-25T16:05:48.809704+0000","last_peered":"2024-01-25T16:05:48.809704+0000","last_clean":"2024-01-25T16:05:48.809704+0000","last_became_active":"2024-01-25T16:05:46.792147+0000","last_became_peered":"2024-01-25T16:05:46.792147+0000","last_unstale":"2024-01-25T16:05:48.809704+0000","last_undegraded":"2024-01-25T16:05:48.809704+0000","last_fullsized":"2024-01-25T16:05:48.809704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:05:08.117739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.810571+0000","last_change":"2024-01-25T16:05:46.808812+0000","last_active":"2024-01-25T16:05:46.810571+0000","last_peered":"2024-01-25T16:05:46.810571+0000","last_clean":"2024-01-25T16:05:46.810571+0000","last_became_active":"2024-01-25T16:05:46.808077+0000","last_became_peered":"2024-01-25T16:05:46.808077+0000","last_unstale":"2024-01-25T16:05:46.810571+0000","last_undegraded":"2024-01-25T16:05:46.810571+0000","last_fullsized":"2024-01-25T16:05:46.810571+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:42:07.536078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.799156+0000","last_change":"2024-01-25T16:05:46.796315+0000","last_active":"2024-01-25T16:05:46.799156+0000","last_peered":"2024-01-25T16:05:46.799156+0000","last_clean":"2024-01-25T16:05:46.799156+0000","last_became_active":"2024-01-25T16:05:46.795998+0000","last_became_peered":"2024-01-25T16:05:46.795998+0000","last_unstale":"2024-01-25T16:05:46.799156+0000","last_undegraded":"2024-01-25T16:05:46.799156+0000","last_fullsized":"2024-01-25T16:05:46.799156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:58:55.788946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807968+0000","last_change":"2024-01-25T16:05:46.798086+0000","last_active":"2024-01-25T16:05:48.807968+0000","last_peered":"2024-01-25T16:05:48.807968+0000","last_clean":"2024-01-25T16:05:48.807968+0000","last_became_active":"2024-01-25T16:05:46.797424+0000","last_became_peered":"2024-01-25T16:05:46.797424+0000","last_unstale":"2024-01-25T16:05:48.807968+0000","last_undegraded":"2024-01-25T16:05:48.807968+0000","last_fullsized":"2024-01-25T16:05:48.807968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:43:04.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809611+0000","last_change":"2024-01-25T16:05:46.799058+0000","last_active":"2024-01-25T16:05:48.809611+0000","last_peered":"2024-01-25T16:05:48.809611+0000","last_clean":"2024-01-25T16:05:48.809611+0000","last_became_active":"2024-01-25T16:05:46.798708+0000","last_became_peered":"2024-01-25T16:05:46.798708+0000","last_unstale":"2024-01-25T16:05:48.809611+0000","last_undegraded":"2024-01-25T16:05:48.809611+0000","last_fullsized":"2024-01-25T16:05:48.809611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:04:48.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809080+0000","last_change":"2024-01-25T16:05:46.791726+0000","last_active":"2024-01-25T16:05:48.809080+0000","last_peered":"2024-01-25T16:05:48.809080+0000","last_clean":"2024-01-25T16:05:48.809080+0000","last_became_active":"2024-01-25T16:05:46.791301+0000","last_became_peered":"2024-01-25T16:05:46.791301+0000","last_unstale":"2024-01-25T16:05:48.809080+0000","last_undegraded":"2024-01-25T16:05:48.809080+0000","last_fullsized":"2024-01-25T16:05:48.809080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:43:25.973750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588280+0000","last_change":"2024-01-25T16:05:46.795523+0000","last_active":"2024-01-25T16:05:48.588280+0000","last_peered":"2024-01-25T16:05:48.588280+0000","last_clean":"2024-01-25T16:05:48.588280+0000","last_became_active":"2024-01-25T16:05:46.794965+0000","last_became_peered":"2024-01-25T16:05:46.794965+0000","last_unstale":"2024-01-25T16:05:48.588280+0000","last_undegraded":"2024-01-25T16:05:48.588280+0000","last_fullsized":"2024-01-25T16:05:48.588280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:13:33.793210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819614+0000","last_change":"2024-01-25T16:05:45.764981+0000","last_active":"2024-01-25T16:05:49.819614+0000","last_peered":"2024-01-25T16:05:49.819614+0000","last_clean":"2024-01-25T16:05:49.819614+0000","last_became_active":"2024-01-25T16:05:45.764616+0000","last_became_peered":"2024-01-25T16:05:45.764616+0000","last_unstale":"2024-01-25T16:05:49.819614+0000","last_undegraded":"2024-01-25T16:05:49.819614+0000","last_fullsized":"2024-01-25T16:05:49.819614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:26:19.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791119+0000","last_change":"2024-01-25T16:05:46.800786+0000","last_active":"2024-01-25T16:05:47.791119+0000","last_peered":"2024-01-25T16:05:47.791119+0000","last_clean":"2024-01-25T16:05:47.791119+0000","last_became_active":"2024-01-25T16:05:46.799666+0000","last_became_peered":"2024-01-25T16:05:46.799666+0000","last_unstale":"2024-01-25T16:05:47.791119+0000","last_undegraded":"2024-01-25T16:05:47.791119+0000","last_fullsized":"2024-01-25T16:05:47.791119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:49:03.942394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791244+0000","last_change":"2024-01-25T16:05:45.773627+0000","last_active":"2024-01-25T16:05:47.791244+0000","last_peered":"2024-01-25T16:05:47.791244+0000","last_clean":"2024-01-25T16:05:47.791244+0000","last_became_active":"2024-01-25T16:05:45.773383+0000","last_became_peered":"2024-01-25T16:05:45.773383+0000","last_unstale":"2024-01-25T16:05:47.791244+0000","last_undegraded":"2024-01-25T16:05:47.791244+0000","last_fullsized":"2024-01-25T16:05:47.791244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:14:22.536001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.803676+0000","last_change":"2024-01-25T16:05:46.801791+0000","last_active":"2024-01-25T16:05:46.803676+0000","last_peered":"2024-01-25T16:05:46.803676+0000","last_clean":"2024-01-25T16:05:46.803676+0000","last_became_active":"2024-01-25T16:05:46.801487+0000","last_became_peered":"2024-01-25T16:05:46.801487+0000","last_unstale":"2024-01-25T16:05:46.803676+0000","last_undegraded":"2024-01-25T16:05:46.803676+0000","last_fullsized":"2024-01-25T16:05:46.803676+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:40:55.379851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.807817+0000","last_change":"2024-01-25T16:05:45.774708+0000","last_active":"2024-01-25T16:05:46.807817+0000","last_peered":"2024-01-25T16:05:46.807817+0000","last_clean":"2024-01-25T16:05:46.807817+0000","last_became_active":"2024-01-25T16:05:45.774401+0000","last_became_peered":"2024-01-25T16:05:45.774401+0000","last_unstale":"2024-01-25T16:05:46.807817+0000","last_undegraded":"2024-01-25T16:05:46.807817+0000","last_fullsized":"2024-01-25T16:05:46.807817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:01:46.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809324+0000","last_change":"2024-01-25T16:05:46.794386+0000","last_active":"2024-01-25T16:05:48.809324+0000","last_peered":"2024-01-25T16:05:48.809324+0000","last_clean":"2024-01-25T16:05:48.809324+0000","last_became_active":"2024-01-25T16:05:46.793909+0000","last_became_peered":"2024-01-25T16:05:46.793909+0000","last_unstale":"2024-01-25T16:05:48.809324+0000","last_undegraded":"2024-01-25T16:05:48.809324+0000","last_fullsized":"2024-01-25T16:05:48.809324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:10:24.736904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.810479+0000","last_change":"2024-01-25T16:05:45.767804+0000","last_active":"2024-01-25T16:05:46.810479+0000","last_peered":"2024-01-25T16:05:46.810479+0000","last_clean":"2024-01-25T16:05:46.810479+0000","last_became_active":"2024-01-25T16:05:45.767491+0000","last_became_peered":"2024-01-25T16:05:45.767491+0000","last_unstale":"2024-01-25T16:05:46.810479+0000","last_undegraded":"2024-01-25T16:05:46.810479+0000","last_fullsized":"2024-01-25T16:05:46.810479+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:20:31.331950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810624+0000","last_change":"2024-01-25T16:05:46.803061+0000","last_active":"2024-01-25T16:05:48.810624+0000","last_peered":"2024-01-25T16:05:48.810624+0000","last_clean":"2024-01-25T16:05:48.810624+0000","last_became_active":"2024-01-25T16:05:46.802756+0000","last_became_peered":"2024-01-25T16:05:46.802756+0000","last_unstale":"2024-01-25T16:05:48.810624+0000","last_undegraded":"2024-01-25T16:05:48.810624+0000","last_fullsized":"2024-01-25T16:05:48.810624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:31:53.107217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:51.056158+0000","last_change":"2024-01-25T16:05:45.764304+0000","last_active":"2024-01-25T16:05:51.056158+0000","last_peered":"2024-01-25T16:05:51.056158+0000","last_clean":"2024-01-25T16:05:51.056158+0000","last_became_active":"2024-01-25T16:05:45.763977+0000","last_became_peered":"2024-01-25T16:05:45.763977+0000","last_unstale":"2024-01-25T16:05:51.056158+0000","last_undegraded":"2024-01-25T16:05:51.056158+0000","last_fullsized":"2024-01-25T16:05:51.056158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:16:07.031015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.805597+0000","last_change":"2024-01-25T16:05:46.802631+0000","last_active":"2024-01-25T16:05:46.805597+0000","last_peered":"2024-01-25T16:05:46.805597+0000","last_clean":"2024-01-25T16:05:46.805597+0000","last_became_active":"2024-01-25T16:05:46.802305+0000","last_became_peered":"2024-01-25T16:05:46.802305+0000","last_unstale":"2024-01-25T16:05:46.805597+0000","last_undegraded":"2024-01-25T16:05:46.805597+0000","last_fullsized":"2024-01-25T16:05:46.805597+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:27:39.594735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819194+0000","last_change":"2024-01-25T16:05:45.762541+0000","last_active":"2024-01-25T16:05:49.819194+0000","last_peered":"2024-01-25T16:05:49.819194+0000","last_clean":"2024-01-25T16:05:49.819194+0000","last_became_active":"2024-01-25T16:05:45.762238+0000","last_became_peered":"2024-01-25T16:05:45.762238+0000","last_unstale":"2024-01-25T16:05:49.819194+0000","last_undegraded":"2024-01-25T16:05:49.819194+0000","last_fullsized":"2024-01-25T16:05:49.819194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:56:41.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808679+0000","last_change":"2024-01-25T16:05:46.794741+0000","last_active":"2024-01-25T16:05:48.808679+0000","last_peered":"2024-01-25T16:05:48.808679+0000","last_clean":"2024-01-25T16:05:48.808679+0000","last_became_active":"2024-01-25T16:05:46.794294+0000","last_became_peered":"2024-01-25T16:05:46.794294+0000","last_unstale":"2024-01-25T16:05:48.808679+0000","last_undegraded":"2024-01-25T16:05:48.808679+0000","last_fullsized":"2024-01-25T16:05:48.808679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:43:00.876478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819633+0000","last_change":"2024-01-25T16:05:45.766535+0000","last_active":"2024-01-25T16:05:49.819633+0000","last_peered":"2024-01-25T16:05:49.819633+0000","last_clean":"2024-01-25T16:05:49.819633+0000","last_became_active":"2024-01-25T16:05:45.765634+0000","last_became_peered":"2024-01-25T16:05:45.765634+0000","last_unstale":"2024-01-25T16:05:49.819633+0000","last_undegraded":"2024-01-25T16:05:49.819633+0000","last_fullsized":"2024-01-25T16:05:49.819633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:43:31.723438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790684+0000","last_change":"2024-01-25T16:05:46.793549+0000","last_active":"2024-01-25T16:05:47.790684+0000","last_peered":"2024-01-25T16:05:47.790684+0000","last_clean":"2024-01-25T16:05:47.790684+0000","last_became_active":"2024-01-25T16:05:46.793217+0000","last_became_peered":"2024-01-25T16:05:46.793217+0000","last_unstale":"2024-01-25T16:05:47.790684+0000","last_undegraded":"2024-01-25T16:05:47.790684+0000","last_fullsized":"2024-01-25T16:05:47.790684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:17:34.531729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807704+0000","last_change":"2024-01-25T16:05:45.768755+0000","last_active":"2024-01-25T16:05:48.807704+0000","last_peered":"2024-01-25T16:05:48.807704+0000","last_clean":"2024-01-25T16:05:48.807704+0000","last_became_active":"2024-01-25T16:05:45.768457+0000","last_became_peered":"2024-01-25T16:05:45.768457+0000","last_unstale":"2024-01-25T16:05:48.807704+0000","last_undegraded":"2024-01-25T16:05:48.807704+0000","last_fullsized":"2024-01-25T16:05:48.807704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:37:46.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808604+0000","last_change":"2024-01-25T16:05:46.792590+0000","last_active":"2024-01-25T16:05:48.808604+0000","last_peered":"2024-01-25T16:05:48.808604+0000","last_clean":"2024-01-25T16:05:48.808604+0000","last_became_active":"2024-01-25T16:05:46.792246+0000","last_became_peered":"2024-01-25T16:05:46.792246+0000","last_unstale":"2024-01-25T16:05:48.808604+0000","last_undegraded":"2024-01-25T16:05:48.808604+0000","last_fullsized":"2024-01-25T16:05:48.808604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:11:31.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814534+0000","last_change":"2024-01-25T16:05:45.765720+0000","last_active":"2024-01-25T16:05:48.814534+0000","last_peered":"2024-01-25T16:05:48.814534+0000","last_clean":"2024-01-25T16:05:48.814534+0000","last_became_active":"2024-01-25T16:05:45.765336+0000","last_became_peered":"2024-01-25T16:05:45.765336+0000","last_unstale":"2024-01-25T16:05:48.814534+0000","last_undegraded":"2024-01-25T16:05:48.814534+0000","last_fullsized":"2024-01-25T16:05:48.814534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:26.016352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809267+0000","last_change":"2024-01-25T16:05:46.795603+0000","last_active":"2024-01-25T16:05:48.809267+0000","last_peered":"2024-01-25T16:05:48.809267+0000","last_clean":"2024-01-25T16:05:48.809267+0000","last_became_active":"2024-01-25T16:05:46.794725+0000","last_became_peered":"2024-01-25T16:05:46.794725+0000","last_unstale":"2024-01-25T16:05:48.809267+0000","last_undegraded":"2024-01-25T16:05:48.809267+0000","last_fullsized":"2024-01-25T16:05:48.809267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:08:28.823911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809381+0000","last_change":"2024-01-25T16:05:45.769249+0000","last_active":"2024-01-25T16:05:48.809381+0000","last_peered":"2024-01-25T16:05:48.809381+0000","last_clean":"2024-01-25T16:05:48.809381+0000","last_became_active":"2024-01-25T16:05:45.768939+0000","last_became_peered":"2024-01-25T16:05:45.768939+0000","last_unstale":"2024-01-25T16:05:48.809381+0000","last_undegraded":"2024-01-25T16:05:48.809381+0000","last_fullsized":"2024-01-25T16:05:48.809381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:32:46.727035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.799691+0000","last_change":"2024-01-25T16:05:46.798868+0000","last_active":"2024-01-25T16:05:46.799691+0000","last_peered":"2024-01-25T16:05:46.799691+0000","last_clean":"2024-01-25T16:05:46.799691+0000","last_became_active":"2024-01-25T16:05:46.798482+0000","last_became_peered":"2024-01-25T16:05:46.798482+0000","last_unstale":"2024-01-25T16:05:46.799691+0000","last_undegraded":"2024-01-25T16:05:46.799691+0000","last_fullsized":"2024-01-25T16:05:46.799691+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:16:09.009107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814003+0000","last_change":"2024-01-25T16:05:45.758681+0000","last_active":"2024-01-25T16:05:48.814003+0000","last_peered":"2024-01-25T16:05:48.814003+0000","last_clean":"2024-01-25T16:05:48.814003+0000","last_became_active":"2024-01-25T16:05:45.758394+0000","last_became_peered":"2024-01-25T16:05:45.758394+0000","last_unstale":"2024-01-25T16:05:48.814003+0000","last_undegraded":"2024-01-25T16:05:48.814003+0000","last_fullsized":"2024-01-25T16:05:48.814003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:55:50.498848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809107+0000","last_change":"2024-01-25T16:05:46.794286+0000","last_active":"2024-01-25T16:05:48.809107+0000","last_peered":"2024-01-25T16:05:48.809107+0000","last_clean":"2024-01-25T16:05:48.809107+0000","last_became_active":"2024-01-25T16:05:46.793949+0000","last_became_peered":"2024-01-25T16:05:46.793949+0000","last_unstale":"2024-01-25T16:05:48.809107+0000","last_undegraded":"2024-01-25T16:05:48.809107+0000","last_fullsized":"2024-01-25T16:05:48.809107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:47:02.196054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.053643+0000","last_change":"2024-01-25T16:05:45.761374+0000","last_active":"2024-01-25T16:05:50.053643+0000","last_peered":"2024-01-25T16:05:50.053643+0000","last_clean":"2024-01-25T16:05:50.053643+0000","last_became_active":"2024-01-25T16:05:45.761060+0000","last_became_peered":"2024-01-25T16:05:45.761060+0000","last_unstale":"2024-01-25T16:05:50.053643+0000","last_undegraded":"2024-01-25T16:05:50.053643+0000","last_fullsized":"2024-01-25T16:05:50.053643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:56:40.591127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.801155+0000","last_change":"2024-01-25T16:05:46.799163+0000","last_active":"2024-01-25T16:05:46.801155+0000","last_peered":"2024-01-25T16:05:46.801155+0000","last_clean":"2024-01-25T16:05:46.801155+0000","last_became_active":"2024-01-25T16:05:46.798771+0000","last_became_peered":"2024-01-25T16:05:46.798771+0000","last_unstale":"2024-01-25T16:05:46.801155+0000","last_undegraded":"2024-01-25T16:05:46.801155+0000","last_fullsized":"2024-01-25T16:05:46.801155+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:18:53.244927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811692+0000","last_change":"2024-01-25T16:05:45.771294+0000","last_active":"2024-01-25T16:05:48.811692+0000","last_peered":"2024-01-25T16:05:48.811692+0000","last_clean":"2024-01-25T16:05:48.811692+0000","last_became_active":"2024-01-25T16:05:45.770986+0000","last_became_peered":"2024-01-25T16:05:45.770986+0000","last_unstale":"2024-01-25T16:05:48.811692+0000","last_undegraded":"2024-01-25T16:05:48.811692+0000","last_fullsized":"2024-01-25T16:05:48.811692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:25:19.430341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809002+0000","last_change":"2024-01-25T16:05:46.792576+0000","last_active":"2024-01-25T16:05:48.809002+0000","last_peered":"2024-01-25T16:05:48.809002+0000","last_clean":"2024-01-25T16:05:48.809002+0000","last_became_active":"2024-01-25T16:05:46.792220+0000","last_became_peered":"2024-01-25T16:05:46.792220+0000","last_unstale":"2024-01-25T16:05:48.809002+0000","last_undegraded":"2024-01-25T16:05:48.809002+0000","last_fullsized":"2024-01-25T16:05:48.809002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:49:31.388289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814185+0000","last_change":"2024-01-25T16:05:45.767265+0000","last_active":"2024-01-25T16:05:48.814185+0000","last_peered":"2024-01-25T16:05:48.814185+0000","last_clean":"2024-01-25T16:05:48.814185+0000","last_became_active":"2024-01-25T16:05:45.766877+0000","last_became_peered":"2024-01-25T16:05:45.766877+0000","last_unstale":"2024-01-25T16:05:48.814185+0000","last_undegraded":"2024-01-25T16:05:48.814185+0000","last_fullsized":"2024-01-25T16:05:48.814185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:47:54.705160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792553+0000","last_change":"2024-01-25T16:05:46.801811+0000","last_active":"2024-01-25T16:05:47.792553+0000","last_peered":"2024-01-25T16:05:47.792553+0000","last_clean":"2024-01-25T16:05:47.792553+0000","last_became_active":"2024-01-25T16:05:46.799546+0000","last_became_peered":"2024-01-25T16:05:46.799546+0000","last_unstale":"2024-01-25T16:05:47.792553+0000","last_undegraded":"2024-01-25T16:05:47.792553+0000","last_fullsized":"2024-01-25T16:05:47.792553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:18:52.900456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.055607+0000","last_change":"2024-01-25T16:05:45.769514+0000","last_active":"2024-01-25T16:05:50.055607+0000","last_peered":"2024-01-25T16:05:50.055607+0000","last_clean":"2024-01-25T16:05:50.055607+0000","last_became_active":"2024-01-25T16:05:45.768497+0000","last_became_peered":"2024-01-25T16:05:45.768497+0000","last_unstale":"2024-01-25T16:05:50.055607+0000","last_undegraded":"2024-01-25T16:05:50.055607+0000","last_fullsized":"2024-01-25T16:05:50.055607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:37.418948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791634+0000","last_change":"2024-01-25T16:05:46.798502+0000","last_active":"2024-01-25T16:05:47.791634+0000","last_peered":"2024-01-25T16:05:47.791634+0000","last_clean":"2024-01-25T16:05:47.791634+0000","last_became_active":"2024-01-25T16:05:46.798203+0000","last_became_peered":"2024-01-25T16:05:46.798203+0000","last_unstale":"2024-01-25T16:05:47.791634+0000","last_undegraded":"2024-01-25T16:05:47.791634+0000","last_fullsized":"2024-01-25T16:05:47.791634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:31:53.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.812042+0000","last_change":"2024-01-25T16:05:45.764827+0000","last_active":"2024-01-25T16:05:46.812042+0000","last_peered":"2024-01-25T16:05:46.812042+0000","last_clean":"2024-01-25T16:05:46.812042+0000","last_became_active":"2024-01-25T16:05:45.764466+0000","last_became_peered":"2024-01-25T16:05:45.764466+0000","last_unstale":"2024-01-25T16:05:46.812042+0000","last_undegraded":"2024-01-25T16:05:46.812042+0000","last_fullsized":"2024-01-25T16:05:46.812042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:57:17.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810077+0000","last_change":"2024-01-25T16:05:46.803960+0000","last_active":"2024-01-25T16:05:48.810077+0000","last_peered":"2024-01-25T16:05:48.810077+0000","last_clean":"2024-01-25T16:05:48.810077+0000","last_became_active":"2024-01-25T16:05:46.803246+0000","last_became_peered":"2024-01-25T16:05:46.803246+0000","last_unstale":"2024-01-25T16:05:48.810077+0000","last_undegraded":"2024-01-25T16:05:48.810077+0000","last_fullsized":"2024-01-25T16:05:48.810077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:42.354813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.811117+0000","last_change":"2024-01-25T16:05:45.765952+0000","last_active":"2024-01-25T16:05:46.811117+0000","last_peered":"2024-01-25T16:05:46.811117+0000","last_clean":"2024-01-25T16:05:46.811117+0000","last_became_active":"2024-01-25T16:05:45.765575+0000","last_became_peered":"2024-01-25T16:05:45.765575+0000","last_unstale":"2024-01-25T16:05:46.811117+0000","last_undegraded":"2024-01-25T16:05:46.811117+0000","last_fullsized":"2024-01-25T16:05:46.811117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:12:07.423245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810082+0000","last_change":"2024-01-25T16:05:46.797577+0000","last_active":"2024-01-25T16:05:48.810082+0000","last_peered":"2024-01-25T16:05:48.810082+0000","last_clean":"2024-01-25T16:05:48.810082+0000","last_became_active":"2024-01-25T16:05:46.796981+0000","last_became_peered":"2024-01-25T16:05:46.796981+0000","last_unstale":"2024-01-25T16:05:48.810082+0000","last_undegraded":"2024-01-25T16:05:48.810082+0000","last_fullsized":"2024-01-25T16:05:48.810082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:09:57.424156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808862+0000","last_change":"2024-01-25T16:05:45.765934+0000","last_active":"2024-01-25T16:05:48.808862+0000","last_peered":"2024-01-25T16:05:48.808862+0000","last_clean":"2024-01-25T16:05:48.808862+0000","last_became_active":"2024-01-25T16:05:45.765583+0000","last_became_peered":"2024-01-25T16:05:45.765583+0000","last_unstale":"2024-01-25T16:05:48.808862+0000","last_undegraded":"2024-01-25T16:05:48.808862+0000","last_fullsized":"2024-01-25T16:05:48.808862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:42:45.814218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808884+0000","last_change":"2024-01-25T16:05:46.794622+0000","last_active":"2024-01-25T16:05:48.808884+0000","last_peered":"2024-01-25T16:05:48.808884+0000","last_clean":"2024-01-25T16:05:48.808884+0000","last_became_active":"2024-01-25T16:05:46.794304+0000","last_became_peered":"2024-01-25T16:05:46.794304+0000","last_unstale":"2024-01-25T16:05:48.808884+0000","last_undegraded":"2024-01-25T16:05:48.808884+0000","last_fullsized":"2024-01-25T16:05:48.808884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:48:31.605501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792643+0000","last_change":"2024-01-25T16:05:45.772412+0000","last_active":"2024-01-25T16:05:47.792643+0000","last_peered":"2024-01-25T16:05:47.792643+0000","last_clean":"2024-01-25T16:05:47.792643+0000","last_became_active":"2024-01-25T16:05:45.772074+0000","last_became_peered":"2024-01-25T16:05:45.772074+0000","last_unstale":"2024-01-25T16:05:47.792643+0000","last_undegraded":"2024-01-25T16:05:47.792643+0000","last_fullsized":"2024-01-25T16:05:47.792643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:42:46.858459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808910+0000","last_change":"2024-01-25T16:05:46.800980+0000","last_active":"2024-01-25T16:05:48.808910+0000","last_peered":"2024-01-25T16:05:48.808910+0000","last_clean":"2024-01-25T16:05:48.808910+0000","last_became_active":"2024-01-25T16:05:46.800689+0000","last_became_peered":"2024-01-25T16:05:46.800689+0000","last_unstale":"2024-01-25T16:05:48.808910+0000","last_undegraded":"2024-01-25T16:05:48.808910+0000","last_fullsized":"2024-01-25T16:05:48.808910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:15:05.197587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807834+0000","last_change":"2024-01-25T16:05:46.802971+0000","last_active":"2024-01-25T16:05:48.807834+0000","last_peered":"2024-01-25T16:05:48.807834+0000","last_clean":"2024-01-25T16:05:48.807834+0000","last_became_active":"2024-01-25T16:05:46.802669+0000","last_became_peered":"2024-01-25T16:05:46.802669+0000","last_unstale":"2024-01-25T16:05:48.807834+0000","last_undegraded":"2024-01-25T16:05:48.807834+0000","last_fullsized":"2024-01-25T16:05:48.807834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:23:01.261255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.052532+0000","last_change":"2024-01-25T16:05:45.763052+0000","last_active":"2024-01-25T16:05:50.052532+0000","last_peered":"2024-01-25T16:05:50.052532+0000","last_clean":"2024-01-25T16:05:50.052532+0000","last_became_active":"2024-01-25T16:05:45.762745+0000","last_became_peered":"2024-01-25T16:05:45.762745+0000","last_unstale":"2024-01-25T16:05:50.052532+0000","last_undegraded":"2024-01-25T16:05:50.052532+0000","last_fullsized":"2024-01-25T16:05:50.052532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:04.783033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790893+0000","last_change":"2024-01-25T16:05:46.799092+0000","last_active":"2024-01-25T16:05:47.790893+0000","last_peered":"2024-01-25T16:05:47.790893+0000","last_clean":"2024-01-25T16:05:47.790893+0000","last_became_active":"2024-01-25T16:05:46.798693+0000","last_became_peered":"2024-01-25T16:05:46.798693+0000","last_unstale":"2024-01-25T16:05:47.790893+0000","last_undegraded":"2024-01-25T16:05:47.790893+0000","last_fullsized":"2024-01-25T16:05:47.790893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:26:29.504645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.050672+0000","last_change":"2024-01-25T16:05:45.760805+0000","last_active":"2024-01-25T16:05:50.050672+0000","last_peered":"2024-01-25T16:05:50.050672+0000","last_clean":"2024-01-25T16:05:50.050672+0000","last_became_active":"2024-01-25T16:05:45.760420+0000","last_became_peered":"2024-01-25T16:05:45.760420+0000","last_unstale":"2024-01-25T16:05:50.050672+0000","last_undegraded":"2024-01-25T16:05:50.050672+0000","last_fullsized":"2024-01-25T16:05:50.050672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:14:16.150271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791853+0000","last_change":"2024-01-25T16:04:34.048543+0000","last_active":"2024-01-25T16:05:47.791853+0000","last_peered":"2024-01-25T16:05:47.791853+0000","last_clean":"2024-01-25T16:05:47.791853+0000","last_became_active":"2024-01-25T16:04:34.048051+0000","last_became_peered":"2024-01-25T16:04:34.048051+0000","last_unstale":"2024-01-25T16:05:47.791853+0000","last_undegraded":"2024-01-25T16:05:47.791853+0000","last_fullsized":"2024-01-25T16:05:47.791853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_clean_scrub_stamp":"2024-01-25T16:01:16.962963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:34:03.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808392+0000","last_change":"2024-01-25T16:05:46.793372+0000","last_active":"2024-01-25T16:05:48.808392+0000","last_peered":"2024-01-25T16:05:48.808392+0000","last_clean":"2024-01-25T16:05:48.808392+0000","last_became_active":"2024-01-25T16:05:46.792312+0000","last_became_peered":"2024-01-25T16:05:46.792312+0000","last_unstale":"2024-01-25T16:05:48.808392+0000","last_undegraded":"2024-01-25T16:05:48.808392+0000","last_fullsized":"2024-01-25T16:05:48.808392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:34:12.261247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.818636+0000","last_change":"2024-01-25T16:05:45.776106+0000","last_active":"2024-01-25T16:05:49.818636+0000","last_peered":"2024-01-25T16:05:49.818636+0000","last_clean":"2024-01-25T16:05:49.818636+0000","last_became_active":"2024-01-25T16:05:45.775880+0000","last_became_peered":"2024-01-25T16:05:45.775880+0000","last_unstale":"2024-01-25T16:05:49.818636+0000","last_undegraded":"2024-01-25T16:05:49.818636+0000","last_fullsized":"2024-01-25T16:05:49.818636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:30:00.130932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.787014+0000","last_change":"2024-01-25T16:05:45.775465+0000","last_active":"2024-01-25T16:05:46.787014+0000","last_peered":"2024-01-25T16:05:46.787014+0000","last_clean":"2024-01-25T16:05:46.787014+0000","last_became_active":"2024-01-25T16:05:45.775068+0000","last_became_peered":"2024-01-25T16:05:45.775068+0000","last_unstale":"2024-01-25T16:05:46.787014+0000","last_undegraded":"2024-01-25T16:05:46.787014+0000","last_fullsized":"2024-01-25T16:05:46.787014+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:29:27.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809682+0000","last_change":"2024-01-25T16:05:46.788033+0000","last_active":"2024-01-25T16:05:48.809682+0000","last_peered":"2024-01-25T16:05:48.809682+0000","last_clean":"2024-01-25T16:05:48.809682+0000","last_became_active":"2024-01-25T16:05:46.787700+0000","last_became_peered":"2024-01-25T16:05:46.787700+0000","last_unstale":"2024-01-25T16:05:48.809682+0000","last_undegraded":"2024-01-25T16:05:48.809682+0000","last_fullsized":"2024-01-25T16:05:48.809682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:41:02.923376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819156+0000","last_change":"2024-01-25T16:05:45.770837+0000","last_active":"2024-01-25T16:05:49.819156+0000","last_peered":"2024-01-25T16:05:49.819156+0000","last_clean":"2024-01-25T16:05:49.819156+0000","last_became_active":"2024-01-25T16:05:45.770501+0000","last_became_peered":"2024-01-25T16:05:45.770501+0000","last_unstale":"2024-01-25T16:05:49.819156+0000","last_undegraded":"2024-01-25T16:05:49.819156+0000","last_fullsized":"2024-01-25T16:05:49.819156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:04:34.900663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791124+0000","last_change":"2024-01-25T16:05:46.810382+0000","last_active":"2024-01-25T16:05:47.791124+0000","last_peered":"2024-01-25T16:05:47.791124+0000","last_clean":"2024-01-25T16:05:47.791124+0000","last_became_active":"2024-01-25T16:05:46.806972+0000","last_became_peered":"2024-01-25T16:05:46.806972+0000","last_unstale":"2024-01-25T16:05:47.791124+0000","last_undegraded":"2024-01-25T16:05:47.791124+0000","last_fullsized":"2024-01-25T16:05:47.791124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:54:44.104250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808231+0000","last_change":"2024-01-25T16:05:45.767147+0000","last_active":"2024-01-25T16:05:48.808231+0000","last_peered":"2024-01-25T16:05:48.808231+0000","last_clean":"2024-01-25T16:05:48.808231+0000","last_became_active":"2024-01-25T16:05:45.766811+0000","last_became_peered":"2024-01-25T16:05:45.766811+0000","last_unstale":"2024-01-25T16:05:48.808231+0000","last_undegraded":"2024-01-25T16:05:48.808231+0000","last_fullsized":"2024-01-25T16:05:48.808231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:51:59.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.799717+0000","last_change":"2024-01-25T16:05:46.798064+0000","last_active":"2024-01-25T16:05:46.799717+0000","last_peered":"2024-01-25T16:05:46.799717+0000","last_clean":"2024-01-25T16:05:46.799717+0000","last_became_active":"2024-01-25T16:05:46.797319+0000","last_became_peered":"2024-01-25T16:05:46.797319+0000","last_unstale":"2024-01-25T16:05:46.799717+0000","last_undegraded":"2024-01-25T16:05:46.799717+0000","last_fullsized":"2024-01-25T16:05:46.799717+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:23:47.128957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812576+0000","last_change":"2024-01-25T16:05:45.761456+0000","last_active":"2024-01-25T16:05:48.812576+0000","last_peered":"2024-01-25T16:05:48.812576+0000","last_clean":"2024-01-25T16:05:48.812576+0000","last_became_active":"2024-01-25T16:05:45.761127+0000","last_became_peered":"2024-01-25T16:05:45.761127+0000","last_unstale":"2024-01-25T16:05:48.812576+0000","last_undegraded":"2024-01-25T16:05:48.812576+0000","last_fullsized":"2024-01-25T16:05:48.812576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:44:00.091230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.797372+0000","last_change":"2024-01-25T16:05:46.792572+0000","last_active":"2024-01-25T16:05:46.797372+0000","last_peered":"2024-01-25T16:05:46.797372+0000","last_clean":"2024-01-25T16:05:46.797372+0000","last_became_active":"2024-01-25T16:05:46.792201+0000","last_became_peered":"2024-01-25T16:05:46.792201+0000","last_unstale":"2024-01-25T16:05:46.797372+0000","last_undegraded":"2024-01-25T16:05:46.797372+0000","last_fullsized":"2024-01-25T16:05:46.797372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:23:27.543183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790635+0000","last_change":"2024-01-25T16:05:45.770183+0000","last_active":"2024-01-25T16:05:47.790635+0000","last_peered":"2024-01-25T16:05:47.790635+0000","last_clean":"2024-01-25T16:05:47.790635+0000","last_became_active":"2024-01-25T16:05:45.769573+0000","last_became_peered":"2024-01-25T16:05:45.769573+0000","last_unstale":"2024-01-25T16:05:47.790635+0000","last_undegraded":"2024-01-25T16:05:47.790635+0000","last_fullsized":"2024-01-25T16:05:47.790635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:39.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808588+0000","last_change":"2024-01-25T16:05:46.795137+0000","last_active":"2024-01-25T16:05:48.808588+0000","last_peered":"2024-01-25T16:05:48.808588+0000","last_clean":"2024-01-25T16:05:48.808588+0000","last_became_active":"2024-01-25T16:05:46.794836+0000","last_became_peered":"2024-01-25T16:05:46.794836+0000","last_unstale":"2024-01-25T16:05:48.808588+0000","last_undegraded":"2024-01-25T16:05:48.808588+0000","last_fullsized":"2024-01-25T16:05:48.808588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:51:27.421203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.587864+0000","last_change":"2024-01-25T16:05:45.765813+0000","last_active":"2024-01-25T16:05:48.587864+0000","last_peered":"2024-01-25T16:05:48.587864+0000","last_clean":"2024-01-25T16:05:48.587864+0000","last_became_active":"2024-01-25T16:05:45.765520+0000","last_became_peered":"2024-01-25T16:05:45.765520+0000","last_unstale":"2024-01-25T16:05:48.587864+0000","last_undegraded":"2024-01-25T16:05:48.587864+0000","last_fullsized":"2024-01-25T16:05:48.587864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:39:45.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791240+0000","last_change":"2024-01-25T16:05:46.805665+0000","last_active":"2024-01-25T16:05:47.791240+0000","last_peered":"2024-01-25T16:05:47.791240+0000","last_clean":"2024-01-25T16:05:47.791240+0000","last_became_active":"2024-01-25T16:05:46.804989+0000","last_became_peered":"2024-01-25T16:05:46.804989+0000","last_unstale":"2024-01-25T16:05:47.791240+0000","last_undegraded":"2024-01-25T16:05:47.791240+0000","last_fullsized":"2024-01-25T16:05:47.791240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:35:07.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.810103+0000","last_change":"2024-01-25T16:05:45.769982+0000","last_active":"2024-01-25T16:05:46.810103+0000","last_peered":"2024-01-25T16:05:46.810103+0000","last_clean":"2024-01-25T16:05:46.810103+0000","last_became_active":"2024-01-25T16:05:45.769617+0000","last_became_peered":"2024-01-25T16:05:45.769617+0000","last_unstale":"2024-01-25T16:05:46.810103+0000","last_undegraded":"2024-01-25T16:05:46.810103+0000","last_fullsized":"2024-01-25T16:05:46.810103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:23.225258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792754+0000","last_change":"2024-01-25T16:05:46.793665+0000","last_active":"2024-01-25T16:05:47.792754+0000","last_peered":"2024-01-25T16:05:47.792754+0000","last_clean":"2024-01-25T16:05:47.792754+0000","last_became_active":"2024-01-25T16:05:46.793335+0000","last_became_peered":"2024-01-25T16:05:46.793335+0000","last_unstale":"2024-01-25T16:05:47.792754+0000","last_undegraded":"2024-01-25T16:05:47.792754+0000","last_fullsized":"2024-01-25T16:05:47.792754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:43:51.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588185+0000","last_change":"2024-01-25T16:05:45.768136+0000","last_active":"2024-01-25T16:05:48.588185+0000","last_peered":"2024-01-25T16:05:48.588185+0000","last_clean":"2024-01-25T16:05:48.588185+0000","last_became_active":"2024-01-25T16:05:45.767847+0000","last_became_peered":"2024-01-25T16:05:45.767847+0000","last_unstale":"2024-01-25T16:05:48.588185+0000","last_undegraded":"2024-01-25T16:05:48.588185+0000","last_fullsized":"2024-01-25T16:05:48.588185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:11:16.154786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792777+0000","last_change":"2024-01-25T16:05:46.794454+0000","last_active":"2024-01-25T16:05:47.792777+0000","last_peered":"2024-01-25T16:05:47.792777+0000","last_clean":"2024-01-25T16:05:47.792777+0000","last_became_active":"2024-01-25T16:05:46.794135+0000","last_became_peered":"2024-01-25T16:05:46.794135+0000","last_unstale":"2024-01-25T16:05:47.792777+0000","last_undegraded":"2024-01-25T16:05:47.792777+0000","last_fullsized":"2024-01-25T16:05:47.792777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:17:56.406013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810489+0000","last_change":"2024-01-25T16:05:45.763459+0000","last_active":"2024-01-25T16:05:48.810489+0000","last_peered":"2024-01-25T16:05:48.810489+0000","last_clean":"2024-01-25T16:05:48.810489+0000","last_became_active":"2024-01-25T16:05:45.763142+0000","last_became_peered":"2024-01-25T16:05:45.763142+0000","last_unstale":"2024-01-25T16:05:48.810489+0000","last_undegraded":"2024-01-25T16:05:48.810489+0000","last_fullsized":"2024-01-25T16:05:48.810489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:52:59.970813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792254+0000","last_change":"2024-01-25T16:05:46.797135+0000","last_active":"2024-01-25T16:05:47.792254+0000","last_peered":"2024-01-25T16:05:47.792254+0000","last_clean":"2024-01-25T16:05:47.792254+0000","last_became_active":"2024-01-25T16:05:46.796838+0000","last_became_peered":"2024-01-25T16:05:46.796838+0000","last_unstale":"2024-01-25T16:05:47.792254+0000","last_undegraded":"2024-01-25T16:05:47.792254+0000","last_fullsized":"2024-01-25T16:05:47.792254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:36:59.271257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809252+0000","last_change":"2024-01-25T16:05:45.762903+0000","last_active":"2024-01-25T16:05:48.809252+0000","last_peered":"2024-01-25T16:05:48.809252+0000","last_clean":"2024-01-25T16:05:48.809252+0000","last_became_active":"2024-01-25T16:05:45.762559+0000","last_became_peered":"2024-01-25T16:05:45.762559+0000","last_unstale":"2024-01-25T16:05:48.809252+0000","last_undegraded":"2024-01-25T16:05:48.809252+0000","last_fullsized":"2024-01-25T16:05:48.809252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:58:57.511914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792478+0000","last_change":"2024-01-25T16:05:46.792890+0000","last_active":"2024-01-25T16:05:47.792478+0000","last_peered":"2024-01-25T16:05:47.792478+0000","last_clean":"2024-01-25T16:05:47.792478+0000","last_became_active":"2024-01-25T16:05:46.792561+0000","last_became_peered":"2024-01-25T16:05:46.792561+0000","last_unstale":"2024-01-25T16:05:47.792478+0000","last_undegraded":"2024-01-25T16:05:47.792478+0000","last_fullsized":"2024-01-25T16:05:47.792478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:55:20.348743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808754+0000","last_change":"2024-01-25T16:05:46.797124+0000","last_active":"2024-01-25T16:05:48.808754+0000","last_peered":"2024-01-25T16:05:48.808754+0000","last_clean":"2024-01-25T16:05:48.808754+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:05:48.808754+0000","last_undegraded":"2024-01-25T16:05:48.808754+0000","last_fullsized":"2024-01-25T16:05:48.808754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:14:38.877808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588048+0000","last_change":"2024-01-25T16:05:46.795688+0000","last_active":"2024-01-25T16:05:48.588048+0000","last_peered":"2024-01-25T16:05:48.588048+0000","last_clean":"2024-01-25T16:05:48.588048+0000","last_became_active":"2024-01-25T16:05:46.794896+0000","last_became_peered":"2024-01-25T16:05:46.794896+0000","last_unstale":"2024-01-25T16:05:48.588048+0000","last_undegraded":"2024-01-25T16:05:48.588048+0000","last_fullsized":"2024-01-25T16:05:48.588048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:15:12.999325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.587971+0000","last_change":"2024-01-25T16:05:46.797036+0000","last_active":"2024-01-25T16:05:48.587971+0000","last_peered":"2024-01-25T16:05:48.587971+0000","last_clean":"2024-01-25T16:05:48.587971+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:05:48.587971+0000","last_undegraded":"2024-01-25T16:05:48.587971+0000","last_fullsized":"2024-01-25T16:05:48.587971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:06:00.095666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-01-25T16:05:46.799211+0000","last_change":"2024-01-25T16:05:46.796367+0000","last_active":"2024-01-25T16:05:46.799211+0000","last_peered":"2024-01-25T16:05:46.799211+0000","last_clean":"2024-01-25T16:05:46.799211+0000","last_became_active":"2024-01-25T16:05:46.795390+0000","last_became_peered":"2024-01-25T16:05:46.795390+0000","last_unstale":"2024-01-25T16:05:46.799211+0000","last_undegraded":"2024-01-25T16:05:46.799211+0000","last_fullsized":"2024-01-25T16:05:46.799211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:36:26.450975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809414+0000","last_change":"2024-01-25T16:05:46.796127+0000","last_active":"2024-01-25T16:05:48.809414+0000","last_peered":"2024-01-25T16:05:48.809414+0000","last_clean":"2024-01-25T16:05:48.809414+0000","last_became_active":"2024-01-25T16:05:46.795667+0000","last_became_peered":"2024-01-25T16:05:46.795667+0000","last_unstale":"2024-01-25T16:05:48.809414+0000","last_undegraded":"2024-01-25T16:05:48.809414+0000","last_fullsized":"2024-01-25T16:05:48.809414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:00:44.411676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809081+0000","last_change":"2024-01-25T16:05:46.796491+0000","last_active":"2024-01-25T16:05:48.809081+0000","last_peered":"2024-01-25T16:05:48.809081+0000","last_clean":"2024-01-25T16:05:48.809081+0000","last_became_active":"2024-01-25T16:05:46.796201+0000","last_became_peered":"2024-01-25T16:05:46.796201+0000","last_unstale":"2024-01-25T16:05:48.809081+0000","last_undegraded":"2024-01-25T16:05:48.809081+0000","last_fullsized":"2024-01-25T16:05:48.809081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:39:42.396263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588407+0000","last_change":"2024-01-25T16:05:46.790680+0000","last_active":"2024-01-25T16:05:48.588407+0000","last_peered":"2024-01-25T16:05:48.588407+0000","last_clean":"2024-01-25T16:05:48.588407+0000","last_became_active":"2024-01-25T16:05:46.790330+0000","last_became_peered":"2024-01-25T16:05:46.790330+0000","last_unstale":"2024-01-25T16:05:48.588407+0000","last_undegraded":"2024-01-25T16:05:48.588407+0000","last_fullsized":"2024-01-25T16:05:48.588407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:33.452484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791035+0000","last_change":"2024-01-25T16:05:46.802958+0000","last_active":"2024-01-25T16:05:47.791035+0000","last_peered":"2024-01-25T16:05:47.791035+0000","last_clean":"2024-01-25T16:05:47.791035+0000","last_became_active":"2024-01-25T16:05:46.802606+0000","last_became_peered":"2024-01-25T16:05:46.802606+0000","last_unstale":"2024-01-25T16:05:47.791035+0000","last_undegraded":"2024-01-25T16:05:47.791035+0000","last_fullsized":"2024-01-25T16:05:47.791035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:57:44.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807776+0000","last_change":"2024-01-25T16:05:46.794974+0000","last_active":"2024-01-25T16:05:48.807776+0000","last_peered":"2024-01-25T16:05:48.807776+0000","last_clean":"2024-01-25T16:05:48.807776+0000","last_became_active":"2024-01-25T16:05:46.794629+0000","last_became_peered":"2024-01-25T16:05:46.794629+0000","last_unstale":"2024-01-25T16:05:48.807776+0000","last_undegraded":"2024-01-25T16:05:48.807776+0000","last_fullsized":"2024-01-25T16:05:48.807776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:19:22.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790649+0000","last_change":"2024-01-25T16:05:46.798725+0000","last_active":"2024-01-25T16:05:47.790649+0000","last_peered":"2024-01-25T16:05:47.790649+0000","last_clean":"2024-01-25T16:05:47.790649+0000","last_became_active":"2024-01-25T16:05:46.798373+0000","last_became_peered":"2024-01-25T16:05:46.798373+0000","last_unstale":"2024-01-25T16:05:47.790649+0000","last_undegraded":"2024-01-25T16:05:47.790649+0000","last_fullsized":"2024-01-25T16:05:47.790649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:55:31.117988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808659+0000","last_change":"2024-01-25T16:05:46.795421+0000","last_active":"2024-01-25T16:05:48.808659+0000","last_peered":"2024-01-25T16:05:48.808659+0000","last_clean":"2024-01-25T16:05:48.808659+0000","last_became_active":"2024-01-25T16:05:46.795068+0000","last_became_peered":"2024-01-25T16:05:46.795068+0000","last_unstale":"2024-01-25T16:05:48.808659+0000","last_undegraded":"2024-01-25T16:05:48.808659+0000","last_fullsized":"2024-01-25T16:05:48.808659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:20:46.490531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809892+0000","last_change":"2024-01-25T16:05:46.802408+0000","last_active":"2024-01-25T16:05:48.809892+0000","last_peered":"2024-01-25T16:05:48.809892+0000","last_clean":"2024-01-25T16:05:48.809892+0000","last_became_active":"2024-01-25T16:05:46.802104+0000","last_became_peered":"2024-01-25T16:05:46.802104+0000","last_unstale":"2024-01-25T16:05:48.809892+0000","last_undegraded":"2024-01-25T16:05:48.809892+0000","last_fullsized":"2024-01-25T16:05:48.809892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:00:11.366282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808629+0000","last_change":"2024-01-25T16:05:46.802553+0000","last_active":"2024-01-25T16:05:48.808629+0000","last_peered":"2024-01-25T16:05:48.808629+0000","last_clean":"2024-01-25T16:05:48.808629+0000","last_became_active":"2024-01-25T16:05:46.802254+0000","last_became_peered":"2024-01-25T16:05:46.802254+0000","last_unstale":"2024-01-25T16:05:48.808629+0000","last_undegraded":"2024-01-25T16:05:48.808629+0000","last_fullsized":"2024-01-25T16:05:48.808629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:37:16.475144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4806,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":22,"num_read_kb":17,"num_write":98,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":30,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188416,"data_stored":13660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":443126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1036102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76900000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.498},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.47699999999999998}]},{"osd":8,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.75800000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59599999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.71099999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":434578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69299999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":0.70399999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.623}]},{"osd":2,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.247,"15min":0.247},"max":{"1min":0.627,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.5}]},{"osd":3,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.41299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.78400000000000003}]},{"osd":5,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65800000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69399999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84599999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.76900000000000002}]},{"osd":9,"last update":"Thu Jan 25 16:05:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.80500000000000005}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.86299999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1034116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.44800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.29399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.497}]},{"osd":2,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68999999999999995,"5min":0.87,"15min":0.87},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82999999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58599999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.51100000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63700000000000001}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61799999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.29899999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.626,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.54300000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96999999999999997,"5min":1.036,"15min":1.036},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.50900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.72599999999999998,"15min":0.70999999999999996},"min":{"1min":0.55100000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63300000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1299999999999999,"5min":1.196,"15min":1.196},"last":0.58599999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:05:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.52800000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:05:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":444156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.35799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 25 16:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.435},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.41899999999999998},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.315}]},{"osd":5,"last update":"Thu Jan 25 16:05:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 25 16:04:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.58899999999999997},"min":{"1min":0.45900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39100000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:05:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57999999999999996}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95599999999999996,"5min":1.155,"15min":1.155},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0169999999999999,"5min":1.073,"15min":1.073},"last":0.81999999999999995}]},{"osd":9,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 25 16:05:34 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738435,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":434064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 25 16:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.996,"15min":0.996},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.502}]},{"osd":3,"last update":"Thu Jan 25 16:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.45500000000000002,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.308},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.44800000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.433}]},{"osd":5,"last update":"Thu Jan 25 16:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59299999999999997,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63400000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:04:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38300000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.45200000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77800000000000002,"15min":0.78600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92000000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61899999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.52800000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:05:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.55300000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":444812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.51900000000000002},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.873,"15min":0.873},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.871,"15min":0.871},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 25 16:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85399999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.437,"15min":0.41999999999999998},"min":{"1min":0.29999999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68000000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.498}]},{"osd":6,"last update":"Thu Jan 25 16:04:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82999999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80400000000000005}]},{"osd":10,"last update":"Thu Jan 25 16:05:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":933888,"data_stored":442648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:04:49 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33400000000000002,"15min":0.32200000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.50600000000000001,"5min":0.755,"15min":0.755},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.33700000000000002},"min":{"1min":0.23599999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.52100000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 25 16:04:49 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.435,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.52900000000000003,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Thu Jan 25 16:04:49 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.55700000000000005,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.56599999999999995,"15min":0.55100000000000005},"min":{"1min":0.436,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.52500000000000002},"min":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.312},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.623}]},{"osd":8,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45300000000000001}]},{"osd":10,"last update":"Thu Jan 25 16:05:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66000000000000003}]}]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.438,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35999999999999999,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.38200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:05:36 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:04:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40799999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73799999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1548288,"data_stored":1034388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.373,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.751,"15min":0.751},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.28100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.751},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.753,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 25 16:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.627}]},{"osd":7,"last update":"Thu Jan 25 16:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70399999999999996}]},{"osd":9,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71799999999999997}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-25T16:05:53.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- 172.21.15.33:0/683363115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc581028c0 msgr2=0x7fbc58109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/683363115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc581028c0 0x7fbc58109150 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009ca0 tx=0x7fbc5002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- 172.21.15.33:0/683363115 shutdown_connections 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/683363115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc581028c0 0x7fbc58109150 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/683363115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc58101f00 0x7fbc58102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/683363115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 0x7fbc58101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- 172.21.15.33:0/683363115 >> 172.21.15.33:0/683363115 conn(0x7fbc580fc2a0 msgr2=0x7fbc580fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- 172.21.15.33:0/683363115 shutdown_connections 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- 172.21.15.33:0/683363115 wait complete. 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 Processor -- start 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.955+0000 7fbc5f25c640 1 -- start start 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 0x7fbc58196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 0x7fbc5819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819e6e0 con 0x7fbc58101f00 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819e850 con 0x7fbc581028c0 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5819ea10 con 0x7fbc58100d10 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50836/0 (socket says 172.21.15.33:50836) 2024-01-25T16:05:53.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 -- 172.21.15.33:0/4145011138 learned_addr learned my addr 172.21.15.33:0/4145011138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5d7f9640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 0x7fbc5819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5cff8640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 0x7fbc58196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 -- 172.21.15.33:0/4145011138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 msgr2=0x7fbc58196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 0x7fbc58196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 -- 172.21.15.33:0/4145011138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 msgr2=0x7fbc5819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 0x7fbc5819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 -- 172.21.15.33:0/4145011138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50009900 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5d7f9640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 0x7fbc5819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc57fff640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c920 tx=0x7fbc4400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44007c70 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc44007dd0 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5819ecf0 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.956+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc440187c0 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.957+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5819f1b0 con 0x7fbc58101f00 2024-01-25T16:05:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.958+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc58107fd0 con 0x7fbc58101f00 2024-01-25T16:05:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.958+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fbc4400f450 con 0x7fbc58101f00 2024-01-25T16:05:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.959+0000 7fbc55ffb640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 0x7fbc480a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.959+0000 7fbc5cff8640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 0x7fbc480a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:53.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.959+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbc440cce40 con 0x7fbc58101f00 2024-01-25T16:05:53.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.959+0000 7fbc5cff8640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 0x7fbc480a3830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005fd0 tx=0x7fbc4c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:53.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:53.964+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbc44096210 con 0x7fbc58101f00 2024-01-25T16:05:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.141+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc5802cbe0 con 0x7fbc480a1370 2024-01-25T16:05:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.151+0000 7fbc55ffb640 1 -- 172.21.15.33:0/4145011138 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336491 (secure 0 0 0) 0x7fbc5802cbe0 con 0x7fbc480a1370 2024-01-25T16:05:54.157 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:54.159 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.154+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 msgr2=0x7fbc480a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.154+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 0x7fbc480a3830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c005fd0 tx=0x7fbc4c005850 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.154+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 msgr2=0x7fbc58197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.154+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c920 tx=0x7fbc4400cde0 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 shutdown_connections 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc581028c0 0x7fbc5819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbc480a1370 0x7fbc480a3830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc58101f00 0x7fbc58197130 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 --2- 172.21.15.33:0/4145011138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbc58100d10 0x7fbc58196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 >> 172.21.15.33:0/4145011138 conn(0x7fbc580fc2a0 msgr2=0x7fbc581051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:54.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 shutdown_connections 2024-01-25T16:05:54.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:54.155+0000 7fbc5f25c640 1 -- 172.21.15.33:0/4145011138 wait complete. 2024-01-25T16:05:54.168 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-01-25T16:05:54.100208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595398,"num_objects":57,"num_object_clones":0,"num_object_copies":171,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":0,"num_read":163,"num_read_kb":141,"num_write":298,"num_write_kb":2976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366164,"kb_used_data":12820,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575660,"statfs":{"total":1151940427776,"available":1151565475840,"internally_reserved":0,"allocated":13127680,"data_stored":7096250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5030,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":27,"num_read_kb":22,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001579"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809413+0000","last_change":"2024-01-25T16:05:46.796305+0000","last_active":"2024-01-25T16:05:48.809413+0000","last_peered":"2024-01-25T16:05:48.809413+0000","last_clean":"2024-01-25T16:05:48.809413+0000","last_became_active":"2024-01-25T16:05:46.795989+0000","last_became_peered":"2024-01-25T16:05:46.795989+0000","last_unstale":"2024-01-25T16:05:48.809413+0000","last_undegraded":"2024-01-25T16:05:48.809413+0000","last_fullsized":"2024-01-25T16:05:48.809413+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:45.764391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812470+0000","last_change":"2024-01-25T16:05:46.798335+0000","last_active":"2024-01-25T16:05:48.812470+0000","last_peered":"2024-01-25T16:05:48.812470+0000","last_clean":"2024-01-25T16:05:48.812470+0000","last_became_active":"2024-01-25T16:05:46.797743+0000","last_became_peered":"2024-01-25T16:05:46.797743+0000","last_unstale":"2024-01-25T16:05:48.812470+0000","last_undegraded":"2024-01-25T16:05:48.812470+0000","last_fullsized":"2024-01-25T16:05:48.812470+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:06:01.335919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811720+0000","last_change":"2024-01-25T16:05:46.803057+0000","last_active":"2024-01-25T16:05:48.811720+0000","last_peered":"2024-01-25T16:05:48.811720+0000","last_clean":"2024-01-25T16:05:48.811720+0000","last_became_active":"2024-01-25T16:05:46.802706+0000","last_became_peered":"2024-01-25T16:05:46.802706+0000","last_unstale":"2024-01-25T16:05:48.811720+0000","last_undegraded":"2024-01-25T16:05:48.811720+0000","last_fullsized":"2024-01-25T16:05:48.811720+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:27.159532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791942+0000","last_change":"2024-01-25T16:05:46.796172+0000","last_active":"2024-01-25T16:05:47.791942+0000","last_peered":"2024-01-25T16:05:47.791942+0000","last_clean":"2024-01-25T16:05:47.791942+0000","last_became_active":"2024-01-25T16:05:46.795848+0000","last_became_peered":"2024-01-25T16:05:46.795848+0000","last_unstale":"2024-01-25T16:05:47.791942+0000","last_undegraded":"2024-01-25T16:05:47.791942+0000","last_fullsized":"2024-01-25T16:05:47.791942+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:41:07.553896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807743+0000","last_change":"2024-01-25T16:05:46.794790+0000","last_active":"2024-01-25T16:05:48.807743+0000","last_peered":"2024-01-25T16:05:48.807743+0000","last_clean":"2024-01-25T16:05:48.807743+0000","last_became_active":"2024-01-25T16:05:46.794464+0000","last_became_peered":"2024-01-25T16:05:46.794464+0000","last_unstale":"2024-01-25T16:05:48.807743+0000","last_undegraded":"2024-01-25T16:05:48.807743+0000","last_fullsized":"2024-01-25T16:05:48.807743+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:19:26.415959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808136+0000","last_change":"2024-01-25T16:05:46.794263+0000","last_active":"2024-01-25T16:05:48.808136+0000","last_peered":"2024-01-25T16:05:48.808136+0000","last_clean":"2024-01-25T16:05:48.808136+0000","last_became_active":"2024-01-25T16:05:46.793598+0000","last_became_peered":"2024-01-25T16:05:46.793598+0000","last_unstale":"2024-01-25T16:05:48.808136+0000","last_undegraded":"2024-01-25T16:05:48.808136+0000","last_fullsized":"2024-01-25T16:05:48.808136+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:47:25.071887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791874+0000","last_change":"2024-01-25T16:05:46.801432+0000","last_active":"2024-01-25T16:05:47.791874+0000","last_peered":"2024-01-25T16:05:47.791874+0000","last_clean":"2024-01-25T16:05:47.791874+0000","last_became_active":"2024-01-25T16:05:46.796673+0000","last_became_peered":"2024-01-25T16:05:46.796673+0000","last_unstale":"2024-01-25T16:05:47.791874+0000","last_undegraded":"2024-01-25T16:05:47.791874+0000","last_fullsized":"2024-01-25T16:05:47.791874+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T04:02:37.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790607+0000","last_change":"2024-01-25T16:05:46.805885+0000","last_active":"2024-01-25T16:05:47.790607+0000","last_peered":"2024-01-25T16:05:47.790607+0000","last_clean":"2024-01-25T16:05:47.790607+0000","last_became_active":"2024-01-25T16:05:46.805359+0000","last_became_peered":"2024-01-25T16:05:46.805359+0000","last_unstale":"2024-01-25T16:05:47.790607+0000","last_undegraded":"2024-01-25T16:05:47.790607+0000","last_fullsized":"2024-01-25T16:05:47.790607+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:44:54.641646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809008+0000","last_change":"2024-01-25T16:05:46.791944+0000","last_active":"2024-01-25T16:05:48.809008+0000","last_peered":"2024-01-25T16:05:48.809008+0000","last_clean":"2024-01-25T16:05:48.809008+0000","last_became_active":"2024-01-25T16:05:46.791637+0000","last_became_peered":"2024-01-25T16:05:46.791637+0000","last_unstale":"2024-01-25T16:05:48.809008+0000","last_undegraded":"2024-01-25T16:05:48.809008+0000","last_fullsized":"2024-01-25T16:05:48.809008+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:54:20.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809017+0000","last_change":"2024-01-25T16:05:46.790961+0000","last_active":"2024-01-25T16:05:48.809017+0000","last_peered":"2024-01-25T16:05:48.809017+0000","last_clean":"2024-01-25T16:05:48.809017+0000","last_became_active":"2024-01-25T16:05:46.790664+0000","last_became_peered":"2024-01-25T16:05:46.790664+0000","last_unstale":"2024-01-25T16:05:48.809017+0000","last_undegraded":"2024-01-25T16:05:48.809017+0000","last_fullsized":"2024-01-25T16:05:48.809017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:24:57.615304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792526+0000","last_change":"2024-01-25T16:05:46.795205+0000","last_active":"2024-01-25T16:05:47.792526+0000","last_peered":"2024-01-25T16:05:47.792526+0000","last_clean":"2024-01-25T16:05:47.792526+0000","last_became_active":"2024-01-25T16:05:46.794894+0000","last_became_peered":"2024-01-25T16:05:46.794894+0000","last_unstale":"2024-01-25T16:05:47.792526+0000","last_undegraded":"2024-01-25T16:05:47.792526+0000","last_fullsized":"2024-01-25T16:05:47.792526+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:57:37.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791897+0000","last_change":"2024-01-25T16:05:46.795473+0000","last_active":"2024-01-25T16:05:47.791897+0000","last_peered":"2024-01-25T16:05:47.791897+0000","last_clean":"2024-01-25T16:05:47.791897+0000","last_became_active":"2024-01-25T16:05:46.795155+0000","last_became_peered":"2024-01-25T16:05:46.795155+0000","last_unstale":"2024-01-25T16:05:47.791897+0000","last_undegraded":"2024-01-25T16:05:47.791897+0000","last_fullsized":"2024-01-25T16:05:47.791897+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:42:25.273806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809704+0000","last_change":"2024-01-25T16:05:46.792550+0000","last_active":"2024-01-25T16:05:48.809704+0000","last_peered":"2024-01-25T16:05:48.809704+0000","last_clean":"2024-01-25T16:05:48.809704+0000","last_became_active":"2024-01-25T16:05:46.792147+0000","last_became_peered":"2024-01-25T16:05:46.792147+0000","last_unstale":"2024-01-25T16:05:48.809704+0000","last_undegraded":"2024-01-25T16:05:48.809704+0000","last_fullsized":"2024-01-25T16:05:48.809704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:05:08.117739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811402+0000","last_change":"2024-01-25T16:05:46.808812+0000","last_active":"2024-01-25T16:05:48.811402+0000","last_peered":"2024-01-25T16:05:48.811402+0000","last_clean":"2024-01-25T16:05:48.811402+0000","last_became_active":"2024-01-25T16:05:46.808077+0000","last_became_peered":"2024-01-25T16:05:46.808077+0000","last_unstale":"2024-01-25T16:05:48.811402+0000","last_undegraded":"2024-01-25T16:05:48.811402+0000","last_fullsized":"2024-01-25T16:05:48.811402+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:56:38.329223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812134+0000","last_change":"2024-01-25T16:05:46.796315+0000","last_active":"2024-01-25T16:05:48.812134+0000","last_peered":"2024-01-25T16:05:48.812134+0000","last_clean":"2024-01-25T16:05:48.812134+0000","last_became_active":"2024-01-25T16:05:46.795998+0000","last_became_peered":"2024-01-25T16:05:46.795998+0000","last_unstale":"2024-01-25T16:05:48.812134+0000","last_undegraded":"2024-01-25T16:05:48.812134+0000","last_fullsized":"2024-01-25T16:05:48.812134+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:44:39.566605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807968+0000","last_change":"2024-01-25T16:05:46.798086+0000","last_active":"2024-01-25T16:05:48.807968+0000","last_peered":"2024-01-25T16:05:48.807968+0000","last_clean":"2024-01-25T16:05:48.807968+0000","last_became_active":"2024-01-25T16:05:46.797424+0000","last_became_peered":"2024-01-25T16:05:46.797424+0000","last_unstale":"2024-01-25T16:05:48.807968+0000","last_undegraded":"2024-01-25T16:05:48.807968+0000","last_fullsized":"2024-01-25T16:05:48.807968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:43:04.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809611+0000","last_change":"2024-01-25T16:05:46.799058+0000","last_active":"2024-01-25T16:05:48.809611+0000","last_peered":"2024-01-25T16:05:48.809611+0000","last_clean":"2024-01-25T16:05:48.809611+0000","last_became_active":"2024-01-25T16:05:46.798708+0000","last_became_peered":"2024-01-25T16:05:46.798708+0000","last_unstale":"2024-01-25T16:05:48.809611+0000","last_undegraded":"2024-01-25T16:05:48.809611+0000","last_fullsized":"2024-01-25T16:05:48.809611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:04:48.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809080+0000","last_change":"2024-01-25T16:05:46.791726+0000","last_active":"2024-01-25T16:05:48.809080+0000","last_peered":"2024-01-25T16:05:48.809080+0000","last_clean":"2024-01-25T16:05:48.809080+0000","last_became_active":"2024-01-25T16:05:46.791301+0000","last_became_peered":"2024-01-25T16:05:46.791301+0000","last_unstale":"2024-01-25T16:05:48.809080+0000","last_undegraded":"2024-01-25T16:05:48.809080+0000","last_fullsized":"2024-01-25T16:05:48.809080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:43:25.973750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588280+0000","last_change":"2024-01-25T16:05:46.795523+0000","last_active":"2024-01-25T16:05:48.588280+0000","last_peered":"2024-01-25T16:05:48.588280+0000","last_clean":"2024-01-25T16:05:48.588280+0000","last_became_active":"2024-01-25T16:05:46.794965+0000","last_became_peered":"2024-01-25T16:05:46.794965+0000","last_unstale":"2024-01-25T16:05:48.588280+0000","last_undegraded":"2024-01-25T16:05:48.588280+0000","last_fullsized":"2024-01-25T16:05:48.588280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:13:33.793210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819614+0000","last_change":"2024-01-25T16:05:45.764981+0000","last_active":"2024-01-25T16:05:49.819614+0000","last_peered":"2024-01-25T16:05:49.819614+0000","last_clean":"2024-01-25T16:05:49.819614+0000","last_became_active":"2024-01-25T16:05:45.764616+0000","last_became_peered":"2024-01-25T16:05:45.764616+0000","last_unstale":"2024-01-25T16:05:49.819614+0000","last_undegraded":"2024-01-25T16:05:49.819614+0000","last_fullsized":"2024-01-25T16:05:49.819614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:26:19.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791119+0000","last_change":"2024-01-25T16:05:46.800786+0000","last_active":"2024-01-25T16:05:47.791119+0000","last_peered":"2024-01-25T16:05:47.791119+0000","last_clean":"2024-01-25T16:05:47.791119+0000","last_became_active":"2024-01-25T16:05:46.799666+0000","last_became_peered":"2024-01-25T16:05:46.799666+0000","last_unstale":"2024-01-25T16:05:47.791119+0000","last_undegraded":"2024-01-25T16:05:47.791119+0000","last_fullsized":"2024-01-25T16:05:47.791119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:49:03.942394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.277790+0000","last_change":"2024-01-25T16:05:45.773627+0000","last_active":"2024-01-25T16:05:48.277790+0000","last_peered":"2024-01-25T16:05:48.277790+0000","last_clean":"2024-01-25T16:05:48.277790+0000","last_became_active":"2024-01-25T16:05:45.773383+0000","last_became_peered":"2024-01-25T16:05:45.773383+0000","last_unstale":"2024-01-25T16:05:48.277790+0000","last_undegraded":"2024-01-25T16:05:48.277790+0000","last_fullsized":"2024-01-25T16:05:48.277790+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:14:22.536001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811959+0000","last_change":"2024-01-25T16:05:46.801791+0000","last_active":"2024-01-25T16:05:48.811959+0000","last_peered":"2024-01-25T16:05:48.811959+0000","last_clean":"2024-01-25T16:05:48.811959+0000","last_became_active":"2024-01-25T16:05:46.801487+0000","last_became_peered":"2024-01-25T16:05:46.801487+0000","last_unstale":"2024-01-25T16:05:48.811959+0000","last_undegraded":"2024-01-25T16:05:48.811959+0000","last_fullsized":"2024-01-25T16:05:48.811959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:45:07.874907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.818144+0000","last_change":"2024-01-25T16:05:45.774708+0000","last_active":"2024-01-25T16:05:49.818144+0000","last_peered":"2024-01-25T16:05:49.818144+0000","last_clean":"2024-01-25T16:05:49.818144+0000","last_became_active":"2024-01-25T16:05:45.774401+0000","last_became_peered":"2024-01-25T16:05:45.774401+0000","last_unstale":"2024-01-25T16:05:49.818144+0000","last_undegraded":"2024-01-25T16:05:49.818144+0000","last_fullsized":"2024-01-25T16:05:49.818144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:01:46.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809324+0000","last_change":"2024-01-25T16:05:46.794386+0000","last_active":"2024-01-25T16:05:48.809324+0000","last_peered":"2024-01-25T16:05:48.809324+0000","last_clean":"2024-01-25T16:05:48.809324+0000","last_became_active":"2024-01-25T16:05:46.793909+0000","last_became_peered":"2024-01-25T16:05:46.793909+0000","last_unstale":"2024-01-25T16:05:48.809324+0000","last_undegraded":"2024-01-25T16:05:48.809324+0000","last_fullsized":"2024-01-25T16:05:48.809324+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:10:24.736904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808979+0000","last_change":"2024-01-25T16:05:45.767804+0000","last_active":"2024-01-25T16:05:48.808979+0000","last_peered":"2024-01-25T16:05:48.808979+0000","last_clean":"2024-01-25T16:05:48.808979+0000","last_became_active":"2024-01-25T16:05:45.767491+0000","last_became_peered":"2024-01-25T16:05:45.767491+0000","last_unstale":"2024-01-25T16:05:48.808979+0000","last_undegraded":"2024-01-25T16:05:48.808979+0000","last_fullsized":"2024-01-25T16:05:48.808979+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:20:31.331950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810624+0000","last_change":"2024-01-25T16:05:46.803061+0000","last_active":"2024-01-25T16:05:48.810624+0000","last_peered":"2024-01-25T16:05:48.810624+0000","last_clean":"2024-01-25T16:05:48.810624+0000","last_became_active":"2024-01-25T16:05:46.802756+0000","last_became_peered":"2024-01-25T16:05:46.802756+0000","last_unstale":"2024-01-25T16:05:48.810624+0000","last_undegraded":"2024-01-25T16:05:48.810624+0000","last_fullsized":"2024-01-25T16:05:48.810624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:31:53.107217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:51.056158+0000","last_change":"2024-01-25T16:05:45.764304+0000","last_active":"2024-01-25T16:05:51.056158+0000","last_peered":"2024-01-25T16:05:51.056158+0000","last_clean":"2024-01-25T16:05:51.056158+0000","last_became_active":"2024-01-25T16:05:45.763977+0000","last_became_peered":"2024-01-25T16:05:45.763977+0000","last_unstale":"2024-01-25T16:05:51.056158+0000","last_undegraded":"2024-01-25T16:05:51.056158+0000","last_fullsized":"2024-01-25T16:05:51.056158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:16:07.031015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807592+0000","last_change":"2024-01-25T16:05:46.802631+0000","last_active":"2024-01-25T16:05:48.807592+0000","last_peered":"2024-01-25T16:05:48.807592+0000","last_clean":"2024-01-25T16:05:48.807592+0000","last_became_active":"2024-01-25T16:05:46.802305+0000","last_became_peered":"2024-01-25T16:05:46.802305+0000","last_unstale":"2024-01-25T16:05:48.807592+0000","last_undegraded":"2024-01-25T16:05:48.807592+0000","last_fullsized":"2024-01-25T16:05:48.807592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:26:59.388515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819194+0000","last_change":"2024-01-25T16:05:45.762541+0000","last_active":"2024-01-25T16:05:49.819194+0000","last_peered":"2024-01-25T16:05:49.819194+0000","last_clean":"2024-01-25T16:05:49.819194+0000","last_became_active":"2024-01-25T16:05:45.762238+0000","last_became_peered":"2024-01-25T16:05:45.762238+0000","last_unstale":"2024-01-25T16:05:49.819194+0000","last_undegraded":"2024-01-25T16:05:49.819194+0000","last_fullsized":"2024-01-25T16:05:49.819194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:56:41.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808679+0000","last_change":"2024-01-25T16:05:46.794741+0000","last_active":"2024-01-25T16:05:48.808679+0000","last_peered":"2024-01-25T16:05:48.808679+0000","last_clean":"2024-01-25T16:05:48.808679+0000","last_became_active":"2024-01-25T16:05:46.794294+0000","last_became_peered":"2024-01-25T16:05:46.794294+0000","last_unstale":"2024-01-25T16:05:48.808679+0000","last_undegraded":"2024-01-25T16:05:48.808679+0000","last_fullsized":"2024-01-25T16:05:48.808679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:43:00.876478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819633+0000","last_change":"2024-01-25T16:05:45.766535+0000","last_active":"2024-01-25T16:05:49.819633+0000","last_peered":"2024-01-25T16:05:49.819633+0000","last_clean":"2024-01-25T16:05:49.819633+0000","last_became_active":"2024-01-25T16:05:45.765634+0000","last_became_peered":"2024-01-25T16:05:45.765634+0000","last_unstale":"2024-01-25T16:05:49.819633+0000","last_undegraded":"2024-01-25T16:05:49.819633+0000","last_fullsized":"2024-01-25T16:05:49.819633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:43:31.723438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790684+0000","last_change":"2024-01-25T16:05:46.793549+0000","last_active":"2024-01-25T16:05:47.790684+0000","last_peered":"2024-01-25T16:05:47.790684+0000","last_clean":"2024-01-25T16:05:47.790684+0000","last_became_active":"2024-01-25T16:05:46.793217+0000","last_became_peered":"2024-01-25T16:05:46.793217+0000","last_unstale":"2024-01-25T16:05:47.790684+0000","last_undegraded":"2024-01-25T16:05:47.790684+0000","last_fullsized":"2024-01-25T16:05:47.790684+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:17:34.531729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807704+0000","last_change":"2024-01-25T16:05:45.768755+0000","last_active":"2024-01-25T16:05:48.807704+0000","last_peered":"2024-01-25T16:05:48.807704+0000","last_clean":"2024-01-25T16:05:48.807704+0000","last_became_active":"2024-01-25T16:05:45.768457+0000","last_became_peered":"2024-01-25T16:05:45.768457+0000","last_unstale":"2024-01-25T16:05:48.807704+0000","last_undegraded":"2024-01-25T16:05:48.807704+0000","last_fullsized":"2024-01-25T16:05:48.807704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:37:46.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808604+0000","last_change":"2024-01-25T16:05:46.792590+0000","last_active":"2024-01-25T16:05:48.808604+0000","last_peered":"2024-01-25T16:05:48.808604+0000","last_clean":"2024-01-25T16:05:48.808604+0000","last_became_active":"2024-01-25T16:05:46.792246+0000","last_became_peered":"2024-01-25T16:05:46.792246+0000","last_unstale":"2024-01-25T16:05:48.808604+0000","last_undegraded":"2024-01-25T16:05:48.808604+0000","last_fullsized":"2024-01-25T16:05:48.808604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:11:31.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814534+0000","last_change":"2024-01-25T16:05:45.765720+0000","last_active":"2024-01-25T16:05:48.814534+0000","last_peered":"2024-01-25T16:05:48.814534+0000","last_clean":"2024-01-25T16:05:48.814534+0000","last_became_active":"2024-01-25T16:05:45.765336+0000","last_became_peered":"2024-01-25T16:05:45.765336+0000","last_unstale":"2024-01-25T16:05:48.814534+0000","last_undegraded":"2024-01-25T16:05:48.814534+0000","last_fullsized":"2024-01-25T16:05:48.814534+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:26.016352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809267+0000","last_change":"2024-01-25T16:05:46.795603+0000","last_active":"2024-01-25T16:05:48.809267+0000","last_peered":"2024-01-25T16:05:48.809267+0000","last_clean":"2024-01-25T16:05:48.809267+0000","last_became_active":"2024-01-25T16:05:46.794725+0000","last_became_peered":"2024-01-25T16:05:46.794725+0000","last_unstale":"2024-01-25T16:05:48.809267+0000","last_undegraded":"2024-01-25T16:05:48.809267+0000","last_fullsized":"2024-01-25T16:05:48.809267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:08:28.823911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809381+0000","last_change":"2024-01-25T16:05:45.769249+0000","last_active":"2024-01-25T16:05:48.809381+0000","last_peered":"2024-01-25T16:05:48.809381+0000","last_clean":"2024-01-25T16:05:48.809381+0000","last_became_active":"2024-01-25T16:05:45.768939+0000","last_became_peered":"2024-01-25T16:05:45.768939+0000","last_unstale":"2024-01-25T16:05:48.809381+0000","last_undegraded":"2024-01-25T16:05:48.809381+0000","last_fullsized":"2024-01-25T16:05:48.809381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:32:46.727035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811829+0000","last_change":"2024-01-25T16:05:46.798868+0000","last_active":"2024-01-25T16:05:48.811829+0000","last_peered":"2024-01-25T16:05:48.811829+0000","last_clean":"2024-01-25T16:05:48.811829+0000","last_became_active":"2024-01-25T16:05:46.798482+0000","last_became_peered":"2024-01-25T16:05:46.798482+0000","last_unstale":"2024-01-25T16:05:48.811829+0000","last_undegraded":"2024-01-25T16:05:48.811829+0000","last_fullsized":"2024-01-25T16:05:48.811829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:35:38.160853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814003+0000","last_change":"2024-01-25T16:05:45.758681+0000","last_active":"2024-01-25T16:05:48.814003+0000","last_peered":"2024-01-25T16:05:48.814003+0000","last_clean":"2024-01-25T16:05:48.814003+0000","last_became_active":"2024-01-25T16:05:45.758394+0000","last_became_peered":"2024-01-25T16:05:45.758394+0000","last_unstale":"2024-01-25T16:05:48.814003+0000","last_undegraded":"2024-01-25T16:05:48.814003+0000","last_fullsized":"2024-01-25T16:05:48.814003+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:55:50.498848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809107+0000","last_change":"2024-01-25T16:05:46.794286+0000","last_active":"2024-01-25T16:05:48.809107+0000","last_peered":"2024-01-25T16:05:48.809107+0000","last_clean":"2024-01-25T16:05:48.809107+0000","last_became_active":"2024-01-25T16:05:46.793949+0000","last_became_peered":"2024-01-25T16:05:46.793949+0000","last_unstale":"2024-01-25T16:05:48.809107+0000","last_undegraded":"2024-01-25T16:05:48.809107+0000","last_fullsized":"2024-01-25T16:05:48.809107+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:47:02.196054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.053643+0000","last_change":"2024-01-25T16:05:45.761374+0000","last_active":"2024-01-25T16:05:50.053643+0000","last_peered":"2024-01-25T16:05:50.053643+0000","last_clean":"2024-01-25T16:05:50.053643+0000","last_became_active":"2024-01-25T16:05:45.761060+0000","last_became_peered":"2024-01-25T16:05:45.761060+0000","last_unstale":"2024-01-25T16:05:50.053643+0000","last_undegraded":"2024-01-25T16:05:50.053643+0000","last_fullsized":"2024-01-25T16:05:50.053643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:56:40.591127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808855+0000","last_change":"2024-01-25T16:05:46.799163+0000","last_active":"2024-01-25T16:05:48.808855+0000","last_peered":"2024-01-25T16:05:48.808855+0000","last_clean":"2024-01-25T16:05:48.808855+0000","last_became_active":"2024-01-25T16:05:46.798771+0000","last_became_peered":"2024-01-25T16:05:46.798771+0000","last_unstale":"2024-01-25T16:05:48.808855+0000","last_undegraded":"2024-01-25T16:05:48.808855+0000","last_fullsized":"2024-01-25T16:05:48.808855+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:30:23.693300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811692+0000","last_change":"2024-01-25T16:05:45.771294+0000","last_active":"2024-01-25T16:05:48.811692+0000","last_peered":"2024-01-25T16:05:48.811692+0000","last_clean":"2024-01-25T16:05:48.811692+0000","last_became_active":"2024-01-25T16:05:45.770986+0000","last_became_peered":"2024-01-25T16:05:45.770986+0000","last_unstale":"2024-01-25T16:05:48.811692+0000","last_undegraded":"2024-01-25T16:05:48.811692+0000","last_fullsized":"2024-01-25T16:05:48.811692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:25:19.430341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809002+0000","last_change":"2024-01-25T16:05:46.792576+0000","last_active":"2024-01-25T16:05:48.809002+0000","last_peered":"2024-01-25T16:05:48.809002+0000","last_clean":"2024-01-25T16:05:48.809002+0000","last_became_active":"2024-01-25T16:05:46.792220+0000","last_became_peered":"2024-01-25T16:05:46.792220+0000","last_unstale":"2024-01-25T16:05:48.809002+0000","last_undegraded":"2024-01-25T16:05:48.809002+0000","last_fullsized":"2024-01-25T16:05:48.809002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:49:31.388289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.814185+0000","last_change":"2024-01-25T16:05:45.767265+0000","last_active":"2024-01-25T16:05:48.814185+0000","last_peered":"2024-01-25T16:05:48.814185+0000","last_clean":"2024-01-25T16:05:48.814185+0000","last_became_active":"2024-01-25T16:05:45.766877+0000","last_became_peered":"2024-01-25T16:05:45.766877+0000","last_unstale":"2024-01-25T16:05:48.814185+0000","last_undegraded":"2024-01-25T16:05:48.814185+0000","last_fullsized":"2024-01-25T16:05:48.814185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:47:54.705160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792553+0000","last_change":"2024-01-25T16:05:46.801811+0000","last_active":"2024-01-25T16:05:47.792553+0000","last_peered":"2024-01-25T16:05:47.792553+0000","last_clean":"2024-01-25T16:05:47.792553+0000","last_became_active":"2024-01-25T16:05:46.799546+0000","last_became_peered":"2024-01-25T16:05:46.799546+0000","last_unstale":"2024-01-25T16:05:47.792553+0000","last_undegraded":"2024-01-25T16:05:47.792553+0000","last_fullsized":"2024-01-25T16:05:47.792553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:18:52.900456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.055607+0000","last_change":"2024-01-25T16:05:45.769514+0000","last_active":"2024-01-25T16:05:50.055607+0000","last_peered":"2024-01-25T16:05:50.055607+0000","last_clean":"2024-01-25T16:05:50.055607+0000","last_became_active":"2024-01-25T16:05:45.768497+0000","last_became_peered":"2024-01-25T16:05:45.768497+0000","last_unstale":"2024-01-25T16:05:50.055607+0000","last_undegraded":"2024-01-25T16:05:50.055607+0000","last_fullsized":"2024-01-25T16:05:50.055607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:37.418948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791634+0000","last_change":"2024-01-25T16:05:46.798502+0000","last_active":"2024-01-25T16:05:47.791634+0000","last_peered":"2024-01-25T16:05:47.791634+0000","last_clean":"2024-01-25T16:05:47.791634+0000","last_became_active":"2024-01-25T16:05:46.798203+0000","last_became_peered":"2024-01-25T16:05:46.798203+0000","last_unstale":"2024-01-25T16:05:47.791634+0000","last_undegraded":"2024-01-25T16:05:47.791634+0000","last_fullsized":"2024-01-25T16:05:47.791634+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:31:53.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819682+0000","last_change":"2024-01-25T16:05:45.764827+0000","last_active":"2024-01-25T16:05:49.819682+0000","last_peered":"2024-01-25T16:05:49.819682+0000","last_clean":"2024-01-25T16:05:49.819682+0000","last_became_active":"2024-01-25T16:05:45.764466+0000","last_became_peered":"2024-01-25T16:05:45.764466+0000","last_unstale":"2024-01-25T16:05:49.819682+0000","last_undegraded":"2024-01-25T16:05:49.819682+0000","last_fullsized":"2024-01-25T16:05:49.819682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:57:17.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810077+0000","last_change":"2024-01-25T16:05:46.803960+0000","last_active":"2024-01-25T16:05:48.810077+0000","last_peered":"2024-01-25T16:05:48.810077+0000","last_clean":"2024-01-25T16:05:48.810077+0000","last_became_active":"2024-01-25T16:05:46.803246+0000","last_became_peered":"2024-01-25T16:05:46.803246+0000","last_unstale":"2024-01-25T16:05:48.810077+0000","last_undegraded":"2024-01-25T16:05:48.810077+0000","last_fullsized":"2024-01-25T16:05:48.810077+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:42.354813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.050719+0000","last_change":"2024-01-25T16:05:45.765952+0000","last_active":"2024-01-25T16:05:50.050719+0000","last_peered":"2024-01-25T16:05:50.050719+0000","last_clean":"2024-01-25T16:05:50.050719+0000","last_became_active":"2024-01-25T16:05:45.765575+0000","last_became_peered":"2024-01-25T16:05:45.765575+0000","last_unstale":"2024-01-25T16:05:50.050719+0000","last_undegraded":"2024-01-25T16:05:50.050719+0000","last_fullsized":"2024-01-25T16:05:50.050719+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:12:07.423245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810082+0000","last_change":"2024-01-25T16:05:46.797577+0000","last_active":"2024-01-25T16:05:48.810082+0000","last_peered":"2024-01-25T16:05:48.810082+0000","last_clean":"2024-01-25T16:05:48.810082+0000","last_became_active":"2024-01-25T16:05:46.796981+0000","last_became_peered":"2024-01-25T16:05:46.796981+0000","last_unstale":"2024-01-25T16:05:48.810082+0000","last_undegraded":"2024-01-25T16:05:48.810082+0000","last_fullsized":"2024-01-25T16:05:48.810082+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:09:57.424156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808862+0000","last_change":"2024-01-25T16:05:45.765934+0000","last_active":"2024-01-25T16:05:48.808862+0000","last_peered":"2024-01-25T16:05:48.808862+0000","last_clean":"2024-01-25T16:05:48.808862+0000","last_became_active":"2024-01-25T16:05:45.765583+0000","last_became_peered":"2024-01-25T16:05:45.765583+0000","last_unstale":"2024-01-25T16:05:48.808862+0000","last_undegraded":"2024-01-25T16:05:48.808862+0000","last_fullsized":"2024-01-25T16:05:48.808862+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:42:45.814218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808884+0000","last_change":"2024-01-25T16:05:46.794622+0000","last_active":"2024-01-25T16:05:48.808884+0000","last_peered":"2024-01-25T16:05:48.808884+0000","last_clean":"2024-01-25T16:05:48.808884+0000","last_became_active":"2024-01-25T16:05:46.794304+0000","last_became_peered":"2024-01-25T16:05:46.794304+0000","last_unstale":"2024-01-25T16:05:48.808884+0000","last_undegraded":"2024-01-25T16:05:48.808884+0000","last_fullsized":"2024-01-25T16:05:48.808884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:48:31.605501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792643+0000","last_change":"2024-01-25T16:05:45.772412+0000","last_active":"2024-01-25T16:05:47.792643+0000","last_peered":"2024-01-25T16:05:47.792643+0000","last_clean":"2024-01-25T16:05:47.792643+0000","last_became_active":"2024-01-25T16:05:45.772074+0000","last_became_peered":"2024-01-25T16:05:45.772074+0000","last_unstale":"2024-01-25T16:05:47.792643+0000","last_undegraded":"2024-01-25T16:05:47.792643+0000","last_fullsized":"2024-01-25T16:05:47.792643+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:42:46.858459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808910+0000","last_change":"2024-01-25T16:05:46.800980+0000","last_active":"2024-01-25T16:05:48.808910+0000","last_peered":"2024-01-25T16:05:48.808910+0000","last_clean":"2024-01-25T16:05:48.808910+0000","last_became_active":"2024-01-25T16:05:46.800689+0000","last_became_peered":"2024-01-25T16:05:46.800689+0000","last_unstale":"2024-01-25T16:05:48.808910+0000","last_undegraded":"2024-01-25T16:05:48.808910+0000","last_fullsized":"2024-01-25T16:05:48.808910+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:15:05.197587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807834+0000","last_change":"2024-01-25T16:05:46.802971+0000","last_active":"2024-01-25T16:05:48.807834+0000","last_peered":"2024-01-25T16:05:48.807834+0000","last_clean":"2024-01-25T16:05:48.807834+0000","last_became_active":"2024-01-25T16:05:46.802669+0000","last_became_peered":"2024-01-25T16:05:46.802669+0000","last_unstale":"2024-01-25T16:05:48.807834+0000","last_undegraded":"2024-01-25T16:05:48.807834+0000","last_fullsized":"2024-01-25T16:05:48.807834+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:23:01.261255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.052532+0000","last_change":"2024-01-25T16:05:45.763052+0000","last_active":"2024-01-25T16:05:50.052532+0000","last_peered":"2024-01-25T16:05:50.052532+0000","last_clean":"2024-01-25T16:05:50.052532+0000","last_became_active":"2024-01-25T16:05:45.762745+0000","last_became_peered":"2024-01-25T16:05:45.762745+0000","last_unstale":"2024-01-25T16:05:50.052532+0000","last_undegraded":"2024-01-25T16:05:50.052532+0000","last_fullsized":"2024-01-25T16:05:50.052532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:04.783033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790893+0000","last_change":"2024-01-25T16:05:46.799092+0000","last_active":"2024-01-25T16:05:47.790893+0000","last_peered":"2024-01-25T16:05:47.790893+0000","last_clean":"2024-01-25T16:05:47.790893+0000","last_became_active":"2024-01-25T16:05:46.798693+0000","last_became_peered":"2024-01-25T16:05:46.798693+0000","last_unstale":"2024-01-25T16:05:47.790893+0000","last_undegraded":"2024-01-25T16:05:47.790893+0000","last_fullsized":"2024-01-25T16:05:47.790893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:26:29.504645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:50.050672+0000","last_change":"2024-01-25T16:05:45.760805+0000","last_active":"2024-01-25T16:05:50.050672+0000","last_peered":"2024-01-25T16:05:50.050672+0000","last_clean":"2024-01-25T16:05:50.050672+0000","last_became_active":"2024-01-25T16:05:45.760420+0000","last_became_peered":"2024-01-25T16:05:45.760420+0000","last_unstale":"2024-01-25T16:05:50.050672+0000","last_undegraded":"2024-01-25T16:05:50.050672+0000","last_fullsized":"2024-01-25T16:05:50.050672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:14:16.150271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791853+0000","last_change":"2024-01-25T16:04:34.048543+0000","last_active":"2024-01-25T16:05:47.791853+0000","last_peered":"2024-01-25T16:05:47.791853+0000","last_clean":"2024-01-25T16:05:47.791853+0000","last_became_active":"2024-01-25T16:04:34.048051+0000","last_became_peered":"2024-01-25T16:04:34.048051+0000","last_unstale":"2024-01-25T16:05:47.791853+0000","last_undegraded":"2024-01-25T16:05:47.791853+0000","last_fullsized":"2024-01-25T16:05:47.791853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_clean_scrub_stamp":"2024-01-25T16:01:16.962963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:34:03.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808392+0000","last_change":"2024-01-25T16:05:46.793372+0000","last_active":"2024-01-25T16:05:48.808392+0000","last_peered":"2024-01-25T16:05:48.808392+0000","last_clean":"2024-01-25T16:05:48.808392+0000","last_became_active":"2024-01-25T16:05:46.792312+0000","last_became_peered":"2024-01-25T16:05:46.792312+0000","last_unstale":"2024-01-25T16:05:48.808392+0000","last_undegraded":"2024-01-25T16:05:48.808392+0000","last_fullsized":"2024-01-25T16:05:48.808392+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:34:12.261247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.818636+0000","last_change":"2024-01-25T16:05:45.776106+0000","last_active":"2024-01-25T16:05:49.818636+0000","last_peered":"2024-01-25T16:05:49.818636+0000","last_clean":"2024-01-25T16:05:49.818636+0000","last_became_active":"2024-01-25T16:05:45.775880+0000","last_became_peered":"2024-01-25T16:05:45.775880+0000","last_unstale":"2024-01-25T16:05:49.818636+0000","last_undegraded":"2024-01-25T16:05:49.818636+0000","last_fullsized":"2024-01-25T16:05:49.818636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:30:00.130932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812766+0000","last_change":"2024-01-25T16:05:45.775465+0000","last_active":"2024-01-25T16:05:48.812766+0000","last_peered":"2024-01-25T16:05:48.812766+0000","last_clean":"2024-01-25T16:05:48.812766+0000","last_became_active":"2024-01-25T16:05:45.775068+0000","last_became_peered":"2024-01-25T16:05:45.775068+0000","last_unstale":"2024-01-25T16:05:48.812766+0000","last_undegraded":"2024-01-25T16:05:48.812766+0000","last_fullsized":"2024-01-25T16:05:48.812766+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:29:27.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809682+0000","last_change":"2024-01-25T16:05:46.788033+0000","last_active":"2024-01-25T16:05:48.809682+0000","last_peered":"2024-01-25T16:05:48.809682+0000","last_clean":"2024-01-25T16:05:48.809682+0000","last_became_active":"2024-01-25T16:05:46.787700+0000","last_became_peered":"2024-01-25T16:05:46.787700+0000","last_unstale":"2024-01-25T16:05:48.809682+0000","last_undegraded":"2024-01-25T16:05:48.809682+0000","last_fullsized":"2024-01-25T16:05:48.809682+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:41:02.923376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:49.819156+0000","last_change":"2024-01-25T16:05:45.770837+0000","last_active":"2024-01-25T16:05:49.819156+0000","last_peered":"2024-01-25T16:05:49.819156+0000","last_clean":"2024-01-25T16:05:49.819156+0000","last_became_active":"2024-01-25T16:05:45.770501+0000","last_became_peered":"2024-01-25T16:05:45.770501+0000","last_unstale":"2024-01-25T16:05:49.819156+0000","last_undegraded":"2024-01-25T16:05:49.819156+0000","last_fullsized":"2024-01-25T16:05:49.819156+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:04:34.900663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791124+0000","last_change":"2024-01-25T16:05:46.810382+0000","last_active":"2024-01-25T16:05:47.791124+0000","last_peered":"2024-01-25T16:05:47.791124+0000","last_clean":"2024-01-25T16:05:47.791124+0000","last_became_active":"2024-01-25T16:05:46.806972+0000","last_became_peered":"2024-01-25T16:05:46.806972+0000","last_unstale":"2024-01-25T16:05:47.791124+0000","last_undegraded":"2024-01-25T16:05:47.791124+0000","last_fullsized":"2024-01-25T16:05:47.791124+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:54:44.104250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808231+0000","last_change":"2024-01-25T16:05:45.767147+0000","last_active":"2024-01-25T16:05:48.808231+0000","last_peered":"2024-01-25T16:05:48.808231+0000","last_clean":"2024-01-25T16:05:48.808231+0000","last_became_active":"2024-01-25T16:05:45.766811+0000","last_became_peered":"2024-01-25T16:05:45.766811+0000","last_unstale":"2024-01-25T16:05:48.808231+0000","last_undegraded":"2024-01-25T16:05:48.808231+0000","last_fullsized":"2024-01-25T16:05:48.808231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:51:59.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812398+0000","last_change":"2024-01-25T16:05:46.798064+0000","last_active":"2024-01-25T16:05:48.812398+0000","last_peered":"2024-01-25T16:05:48.812398+0000","last_clean":"2024-01-25T16:05:48.812398+0000","last_became_active":"2024-01-25T16:05:46.797319+0000","last_became_peered":"2024-01-25T16:05:46.797319+0000","last_unstale":"2024-01-25T16:05:48.812398+0000","last_undegraded":"2024-01-25T16:05:48.812398+0000","last_fullsized":"2024-01-25T16:05:48.812398+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:02.676568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.812576+0000","last_change":"2024-01-25T16:05:45.761456+0000","last_active":"2024-01-25T16:05:48.812576+0000","last_peered":"2024-01-25T16:05:48.812576+0000","last_clean":"2024-01-25T16:05:48.812576+0000","last_became_active":"2024-01-25T16:05:45.761127+0000","last_became_peered":"2024-01-25T16:05:45.761127+0000","last_unstale":"2024-01-25T16:05:48.812576+0000","last_undegraded":"2024-01-25T16:05:48.812576+0000","last_fullsized":"2024-01-25T16:05:48.812576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:44:00.091230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807445+0000","last_change":"2024-01-25T16:05:46.792572+0000","last_active":"2024-01-25T16:05:48.807445+0000","last_peered":"2024-01-25T16:05:48.807445+0000","last_clean":"2024-01-25T16:05:48.807445+0000","last_became_active":"2024-01-25T16:05:46.792201+0000","last_became_peered":"2024-01-25T16:05:46.792201+0000","last_unstale":"2024-01-25T16:05:48.807445+0000","last_undegraded":"2024-01-25T16:05:48.807445+0000","last_fullsized":"2024-01-25T16:05:48.807445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:17:29.636840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810259+0000","last_change":"2024-01-25T16:05:45.770183+0000","last_active":"2024-01-25T16:05:48.810259+0000","last_peered":"2024-01-25T16:05:48.810259+0000","last_clean":"2024-01-25T16:05:48.810259+0000","last_became_active":"2024-01-25T16:05:45.769573+0000","last_became_peered":"2024-01-25T16:05:45.769573+0000","last_unstale":"2024-01-25T16:05:48.810259+0000","last_undegraded":"2024-01-25T16:05:48.810259+0000","last_fullsized":"2024-01-25T16:05:48.810259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:39.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808588+0000","last_change":"2024-01-25T16:05:46.795137+0000","last_active":"2024-01-25T16:05:48.808588+0000","last_peered":"2024-01-25T16:05:48.808588+0000","last_clean":"2024-01-25T16:05:48.808588+0000","last_became_active":"2024-01-25T16:05:46.794836+0000","last_became_peered":"2024-01-25T16:05:46.794836+0000","last_unstale":"2024-01-25T16:05:48.808588+0000","last_undegraded":"2024-01-25T16:05:48.808588+0000","last_fullsized":"2024-01-25T16:05:48.808588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:51:27.421203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.587864+0000","last_change":"2024-01-25T16:05:45.765813+0000","last_active":"2024-01-25T16:05:48.587864+0000","last_peered":"2024-01-25T16:05:48.587864+0000","last_clean":"2024-01-25T16:05:48.587864+0000","last_became_active":"2024-01-25T16:05:45.765520+0000","last_became_peered":"2024-01-25T16:05:45.765520+0000","last_unstale":"2024-01-25T16:05:48.587864+0000","last_undegraded":"2024-01-25T16:05:48.587864+0000","last_fullsized":"2024-01-25T16:05:48.587864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:39:45.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791240+0000","last_change":"2024-01-25T16:05:46.805665+0000","last_active":"2024-01-25T16:05:47.791240+0000","last_peered":"2024-01-25T16:05:47.791240+0000","last_clean":"2024-01-25T16:05:47.791240+0000","last_became_active":"2024-01-25T16:05:46.804989+0000","last_became_peered":"2024-01-25T16:05:46.804989+0000","last_unstale":"2024-01-25T16:05:47.791240+0000","last_undegraded":"2024-01-25T16:05:47.791240+0000","last_fullsized":"2024-01-25T16:05:47.791240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:35:07.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.813370+0000","last_change":"2024-01-25T16:05:45.769982+0000","last_active":"2024-01-25T16:05:48.813370+0000","last_peered":"2024-01-25T16:05:48.813370+0000","last_clean":"2024-01-25T16:05:48.813370+0000","last_became_active":"2024-01-25T16:05:45.769617+0000","last_became_peered":"2024-01-25T16:05:45.769617+0000","last_unstale":"2024-01-25T16:05:48.813370+0000","last_undegraded":"2024-01-25T16:05:48.813370+0000","last_fullsized":"2024-01-25T16:05:48.813370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:23.225258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792754+0000","last_change":"2024-01-25T16:05:46.793665+0000","last_active":"2024-01-25T16:05:47.792754+0000","last_peered":"2024-01-25T16:05:47.792754+0000","last_clean":"2024-01-25T16:05:47.792754+0000","last_became_active":"2024-01-25T16:05:46.793335+0000","last_became_peered":"2024-01-25T16:05:46.793335+0000","last_unstale":"2024-01-25T16:05:47.792754+0000","last_undegraded":"2024-01-25T16:05:47.792754+0000","last_fullsized":"2024-01-25T16:05:47.792754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:43:51.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588185+0000","last_change":"2024-01-25T16:05:45.768136+0000","last_active":"2024-01-25T16:05:48.588185+0000","last_peered":"2024-01-25T16:05:48.588185+0000","last_clean":"2024-01-25T16:05:48.588185+0000","last_became_active":"2024-01-25T16:05:45.767847+0000","last_became_peered":"2024-01-25T16:05:45.767847+0000","last_unstale":"2024-01-25T16:05:48.588185+0000","last_undegraded":"2024-01-25T16:05:48.588185+0000","last_fullsized":"2024-01-25T16:05:48.588185+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:11:16.154786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792777+0000","last_change":"2024-01-25T16:05:46.794454+0000","last_active":"2024-01-25T16:05:47.792777+0000","last_peered":"2024-01-25T16:05:47.792777+0000","last_clean":"2024-01-25T16:05:47.792777+0000","last_became_active":"2024-01-25T16:05:46.794135+0000","last_became_peered":"2024-01-25T16:05:46.794135+0000","last_unstale":"2024-01-25T16:05:47.792777+0000","last_undegraded":"2024-01-25T16:05:47.792777+0000","last_fullsized":"2024-01-25T16:05:47.792777+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:17:56.406013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.810489+0000","last_change":"2024-01-25T16:05:45.763459+0000","last_active":"2024-01-25T16:05:48.810489+0000","last_peered":"2024-01-25T16:05:48.810489+0000","last_clean":"2024-01-25T16:05:48.810489+0000","last_became_active":"2024-01-25T16:05:45.763142+0000","last_became_peered":"2024-01-25T16:05:45.763142+0000","last_unstale":"2024-01-25T16:05:48.810489+0000","last_undegraded":"2024-01-25T16:05:48.810489+0000","last_fullsized":"2024-01-25T16:05:48.810489+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:52:59.970813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792254+0000","last_change":"2024-01-25T16:05:46.797135+0000","last_active":"2024-01-25T16:05:47.792254+0000","last_peered":"2024-01-25T16:05:47.792254+0000","last_clean":"2024-01-25T16:05:47.792254+0000","last_became_active":"2024-01-25T16:05:46.796838+0000","last_became_peered":"2024-01-25T16:05:46.796838+0000","last_unstale":"2024-01-25T16:05:47.792254+0000","last_undegraded":"2024-01-25T16:05:47.792254+0000","last_fullsized":"2024-01-25T16:05:47.792254+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:36:59.271257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809252+0000","last_change":"2024-01-25T16:05:45.762903+0000","last_active":"2024-01-25T16:05:48.809252+0000","last_peered":"2024-01-25T16:05:48.809252+0000","last_clean":"2024-01-25T16:05:48.809252+0000","last_became_active":"2024-01-25T16:05:45.762559+0000","last_became_peered":"2024-01-25T16:05:45.762559+0000","last_unstale":"2024-01-25T16:05:48.809252+0000","last_undegraded":"2024-01-25T16:05:48.809252+0000","last_fullsized":"2024-01-25T16:05:48.809252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:58:57.511914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.792478+0000","last_change":"2024-01-25T16:05:46.792890+0000","last_active":"2024-01-25T16:05:47.792478+0000","last_peered":"2024-01-25T16:05:47.792478+0000","last_clean":"2024-01-25T16:05:47.792478+0000","last_became_active":"2024-01-25T16:05:46.792561+0000","last_became_peered":"2024-01-25T16:05:46.792561+0000","last_unstale":"2024-01-25T16:05:47.792478+0000","last_undegraded":"2024-01-25T16:05:47.792478+0000","last_fullsized":"2024-01-25T16:05:47.792478+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:55:20.348743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808754+0000","last_change":"2024-01-25T16:05:46.797124+0000","last_active":"2024-01-25T16:05:48.808754+0000","last_peered":"2024-01-25T16:05:48.808754+0000","last_clean":"2024-01-25T16:05:48.808754+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:05:48.808754+0000","last_undegraded":"2024-01-25T16:05:48.808754+0000","last_fullsized":"2024-01-25T16:05:48.808754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:14:38.877808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588048+0000","last_change":"2024-01-25T16:05:46.795688+0000","last_active":"2024-01-25T16:05:48.588048+0000","last_peered":"2024-01-25T16:05:48.588048+0000","last_clean":"2024-01-25T16:05:48.588048+0000","last_became_active":"2024-01-25T16:05:46.794896+0000","last_became_peered":"2024-01-25T16:05:46.794896+0000","last_unstale":"2024-01-25T16:05:48.588048+0000","last_undegraded":"2024-01-25T16:05:48.588048+0000","last_fullsized":"2024-01-25T16:05:48.588048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:15:12.999325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.587971+0000","last_change":"2024-01-25T16:05:46.797036+0000","last_active":"2024-01-25T16:05:48.587971+0000","last_peered":"2024-01-25T16:05:48.587971+0000","last_clean":"2024-01-25T16:05:48.587971+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:05:48.587971+0000","last_undegraded":"2024-01-25T16:05:48.587971+0000","last_fullsized":"2024-01-25T16:05:48.587971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:06:00.095666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.811673+0000","last_change":"2024-01-25T16:05:46.796367+0000","last_active":"2024-01-25T16:05:48.811673+0000","last_peered":"2024-01-25T16:05:48.811673+0000","last_clean":"2024-01-25T16:05:48.811673+0000","last_became_active":"2024-01-25T16:05:46.795390+0000","last_became_peered":"2024-01-25T16:05:46.795390+0000","last_unstale":"2024-01-25T16:05:48.811673+0000","last_undegraded":"2024-01-25T16:05:48.811673+0000","last_fullsized":"2024-01-25T16:05:48.811673+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:35:41.524542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809414+0000","last_change":"2024-01-25T16:05:46.796127+0000","last_active":"2024-01-25T16:05:48.809414+0000","last_peered":"2024-01-25T16:05:48.809414+0000","last_clean":"2024-01-25T16:05:48.809414+0000","last_became_active":"2024-01-25T16:05:46.795667+0000","last_became_peered":"2024-01-25T16:05:46.795667+0000","last_unstale":"2024-01-25T16:05:48.809414+0000","last_undegraded":"2024-01-25T16:05:48.809414+0000","last_fullsized":"2024-01-25T16:05:48.809414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:00:44.411676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809081+0000","last_change":"2024-01-25T16:05:46.796491+0000","last_active":"2024-01-25T16:05:48.809081+0000","last_peered":"2024-01-25T16:05:48.809081+0000","last_clean":"2024-01-25T16:05:48.809081+0000","last_became_active":"2024-01-25T16:05:46.796201+0000","last_became_peered":"2024-01-25T16:05:46.796201+0000","last_unstale":"2024-01-25T16:05:48.809081+0000","last_undegraded":"2024-01-25T16:05:48.809081+0000","last_fullsized":"2024-01-25T16:05:48.809081+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:39:42.396263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.588407+0000","last_change":"2024-01-25T16:05:46.790680+0000","last_active":"2024-01-25T16:05:48.588407+0000","last_peered":"2024-01-25T16:05:48.588407+0000","last_clean":"2024-01-25T16:05:48.588407+0000","last_became_active":"2024-01-25T16:05:46.790330+0000","last_became_peered":"2024-01-25T16:05:46.790330+0000","last_unstale":"2024-01-25T16:05:48.588407+0000","last_undegraded":"2024-01-25T16:05:48.588407+0000","last_fullsized":"2024-01-25T16:05:48.588407+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:33.452484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.791035+0000","last_change":"2024-01-25T16:05:46.802958+0000","last_active":"2024-01-25T16:05:47.791035+0000","last_peered":"2024-01-25T16:05:47.791035+0000","last_clean":"2024-01-25T16:05:47.791035+0000","last_became_active":"2024-01-25T16:05:46.802606+0000","last_became_peered":"2024-01-25T16:05:46.802606+0000","last_unstale":"2024-01-25T16:05:47.791035+0000","last_undegraded":"2024-01-25T16:05:47.791035+0000","last_fullsized":"2024-01-25T16:05:47.791035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:57:44.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.807776+0000","last_change":"2024-01-25T16:05:46.794974+0000","last_active":"2024-01-25T16:05:48.807776+0000","last_peered":"2024-01-25T16:05:48.807776+0000","last_clean":"2024-01-25T16:05:48.807776+0000","last_became_active":"2024-01-25T16:05:46.794629+0000","last_became_peered":"2024-01-25T16:05:46.794629+0000","last_unstale":"2024-01-25T16:05:48.807776+0000","last_undegraded":"2024-01-25T16:05:48.807776+0000","last_fullsized":"2024-01-25T16:05:48.807776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:19:22.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:47.790649+0000","last_change":"2024-01-25T16:05:46.798725+0000","last_active":"2024-01-25T16:05:47.790649+0000","last_peered":"2024-01-25T16:05:47.790649+0000","last_clean":"2024-01-25T16:05:47.790649+0000","last_became_active":"2024-01-25T16:05:46.798373+0000","last_became_peered":"2024-01-25T16:05:46.798373+0000","last_unstale":"2024-01-25T16:05:47.790649+0000","last_undegraded":"2024-01-25T16:05:47.790649+0000","last_fullsized":"2024-01-25T16:05:47.790649+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:55:31.117988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808659+0000","last_change":"2024-01-25T16:05:46.795421+0000","last_active":"2024-01-25T16:05:48.808659+0000","last_peered":"2024-01-25T16:05:48.808659+0000","last_clean":"2024-01-25T16:05:48.808659+0000","last_became_active":"2024-01-25T16:05:46.795068+0000","last_became_peered":"2024-01-25T16:05:46.795068+0000","last_unstale":"2024-01-25T16:05:48.808659+0000","last_undegraded":"2024-01-25T16:05:48.808659+0000","last_fullsized":"2024-01-25T16:05:48.808659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:20:46.490531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.809892+0000","last_change":"2024-01-25T16:05:46.802408+0000","last_active":"2024-01-25T16:05:48.809892+0000","last_peered":"2024-01-25T16:05:48.809892+0000","last_clean":"2024-01-25T16:05:48.809892+0000","last_became_active":"2024-01-25T16:05:46.802104+0000","last_became_peered":"2024-01-25T16:05:46.802104+0000","last_unstale":"2024-01-25T16:05:48.809892+0000","last_undegraded":"2024-01-25T16:05:48.809892+0000","last_fullsized":"2024-01-25T16:05:48.809892+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:00:11.366282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-01-25T16:05:48.808629+0000","last_change":"2024-01-25T16:05:46.802553+0000","last_active":"2024-01-25T16:05:48.808629+0000","last_peered":"2024-01-25T16:05:48.808629+0000","last_clean":"2024-01-25T16:05:48.808629+0000","last_became_active":"2024-01-25T16:05:46.802254+0000","last_became_peered":"2024-01-25T16:05:46.802254+0000","last_unstale":"2024-01-25T16:05:48.808629+0000","last_undegraded":"2024-01-25T16:05:48.808629+0000","last_fullsized":"2024-01-25T16:05:48.808629+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:37:16.475144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":5030,"num_objects":55,"num_object_clones":0,"num_object_copies":165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":27,"num_read_kb":22,"num_write":117,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":15362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":443126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]}]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28544,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716608,"statfs":{"total":95995035648,"available":95965806592,"internally_reserved":0,"allocated":1564672,"data_stored":1036102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":1,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.76900000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.38300000000000001}]},{"osd":5,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72299999999999998}]},{"osd":6,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.70599999999999996}]},{"osd":7,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.498},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.47699999999999998}]},{"osd":8,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.75800000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.74199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]}]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":946176,"data_stored":443362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.40400000000000003}]},{"osd":1,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.63300000000000001}]},{"osd":3,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.42699999999999999}]},{"osd":4,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.64400000000000002}]},{"osd":6,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44600000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.59599999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.67500000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.71099999999999997}]}]},{"osd":8,"up_from":60,"seq":257698037789,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":443282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69299999999999995,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72099999999999997,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73299999999999998,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.59299999999999997,"15min":0.59899999999999998},"min":{"1min":0.438,"5min":0.377,"15min":0.377},"max":{"1min":0.70399999999999996,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.82999999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.44500000000000001,"15min":0.44700000000000001},"min":{"1min":0.30299999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.313,"5min":0.247,"15min":0.247},"max":{"1min":0.627,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.56200000000000006,"15min":0.56999999999999995},"min":{"1min":0.39900000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73599999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.41299999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78600000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.41299999999999998}]},{"osd":4,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.434,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.754,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.59899999999999998,"15min":0.59699999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.51700000000000002,"15min":0.52200000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65800000000000003,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.32900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.64300000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55700000000000005}]},{"osd":6,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61799999999999999,"15min":0.624},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.64500000000000002,"15min":0.65400000000000003},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81000000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.71099999999999997}]},{"osd":7,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67200000000000004,"15min":0.67400000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.84599999999999997,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.67400000000000004,"15min":0.68000000000000005},"min":{"1min":0.503,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.79700000000000004}]},{"osd":9,"last update":"Thu Jan 25 16:05:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.85899999999999999}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.76500000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]}]},{"osd":7,"up_from":52,"seq":223338299424,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32672,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712480,"statfs":{"total":95995035648,"available":95961579520,"internally_reserved":0,"allocated":1531904,"data_stored":1034116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.44800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.40600000000000003,"15min":0.39500000000000002},"min":{"1min":0.28599999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.39600000000000002},"min":{"1min":0.29399999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.497}]},{"osd":2,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.68600000000000005,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.68999999999999995,"5min":0.87,"15min":0.87},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.53200000000000003,"15min":0.52100000000000002},"min":{"1min":0.42999999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.48499999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57299999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.49299999999999999,"15min":0.48499999999999999},"min":{"1min":0.36499999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.498,"15min":0.496},"min":{"1min":0.34399999999999997,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.82999999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.59899999999999998,"15min":0.58999999999999997},"min":{"1min":0.45300000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.439}]},{"osd":6,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.64300000000000002,"15min":0.629},"min":{"1min":0.50800000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63300000000000001,"15min":0.624},"min":{"1min":0.45900000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58599999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.52500000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.51100000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63700000000000001}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.54800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":442634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61799999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.42099999999999999},"min":{"1min":0.29899999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.77000000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.626,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.622,"15min":0.60899999999999999},"min":{"1min":0.41699999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.434,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.54300000000000004,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54500000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51400000000000001,"15min":0.51000000000000001},"min":{"1min":0.39800000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.93799999999999994,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.53100000000000003,"15min":0.52200000000000002},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.58899999999999997,"15min":0.59099999999999997},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.44700000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96999999999999997,"5min":1.036,"15min":1.036},"last":0.69499999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.64600000000000002,"15min":0.64000000000000001},"min":{"1min":0.50900000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.83999999999999997,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.56699999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.72599999999999998,"15min":0.70999999999999996},"min":{"1min":0.55100000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63300000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.77400000000000002,"15min":0.77200000000000002},"min":{"1min":0.51800000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.1299999999999999,"5min":1.196,"15min":1.196},"last":0.58599999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:05:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.52800000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:05:31 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.71499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":444156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.503,"15min":0.48799999999999999},"min":{"1min":0.40600000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49399999999999999,"15min":0.47999999999999998},"min":{"1min":0.35799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 25 16:05:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.40600000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:05:47 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.56499999999999995,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.82599999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.435},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.57699999999999996,"15min":0.55700000000000005},"min":{"1min":0.39600000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.81699999999999995,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.42299999999999999,"15min":0.41899999999999998},"min":{"1min":0.27800000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.31,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.315}]},{"osd":5,"last update":"Thu Jan 25 16:05:34 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.60399999999999998,"15min":0.57799999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.58999999999999997,"15min":0.57399999999999995},"min":{"1min":0.45900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 25 16:04:57 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60799999999999998,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.60399999999999998,"15min":0.58899999999999997},"min":{"1min":0.45900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.39100000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:05:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.54600000000000004,"15min":0.53500000000000003},"min":{"1min":0.41799999999999998,"5min":0.313,"15min":0.313},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.38700000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57999999999999996}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.95599999999999996,"5min":1.155,"15min":1.155},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.70999999999999996,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.0169999999999999,"5min":1.073,"15min":1.073},"last":0.81999999999999995}]},{"osd":9,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.78200000000000003}]},{"osd":10,"last update":"Thu Jan 25 16:05:34 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.626}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":444022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 25 16:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52400000000000002,"15min":0.51600000000000001},"min":{"1min":0.39700000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.74299999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53000000000000003,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.77400000000000002,"5min":0.996,"15min":0.996},"last":0.53800000000000003}]},{"osd":2,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55000000000000004,"15min":0.51800000000000002},"min":{"1min":0.39100000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.54700000000000004,"15min":0.52100000000000002},"min":{"1min":0.36699999999999999,"5min":0.308,"15min":0.308},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:04:40 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.45500000000000002,"15min":0.42899999999999999},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.46000000000000002,"15min":0.44800000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.68999999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.45800000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.54700000000000004,"15min":0.52800000000000002},"min":{"1min":0.44800000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.56200000000000006,"15min":0.54200000000000004},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:05:35 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53400000000000003},"min":{"1min":0.37,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.59299999999999997,"15min":0.56999999999999995},"min":{"1min":0.36199999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.629}]},{"osd":6,"last update":"Thu Jan 25 16:04:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55800000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.45200000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:05:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.77800000000000002,"15min":0.78600000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.92000000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.61799999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:05:05 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.55100000000000005}]},{"osd":10,"last update":"Thu Jan 25 16:05:32 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.64800000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":444812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.375,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75600000000000001,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.51100000000000001,"15min":0.51900000000000002},"min":{"1min":0.34699999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:05:25 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.52000000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.873,"15min":0.873},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.33200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.76100000000000001}]},{"osd":3,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.44400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77400000000000002,"5min":0.871,"15min":0.871},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58799999999999997,"15min":0.57099999999999995},"min":{"1min":0.38200000000000001,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61599999999999999}]},{"osd":4,"last update":"Thu Jan 25 16:05:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.35899999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.77400000000000002,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.437,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85399999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.437,"15min":0.41999999999999998},"min":{"1min":0.29999999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.68799999999999994,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.443,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.68000000000000005,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.498}]},{"osd":6,"last update":"Thu Jan 25 16:04:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58599999999999997,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.82999999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.78700000000000003}]},{"osd":9,"last update":"Thu Jan 25 16:05:06 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.80400000000000005}]},{"osd":10,"last update":"Thu Jan 25 16:05:31 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.82299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93600000000000005}]}]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":950272,"data_stored":443062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:51 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33800000000000002,"15min":0.32300000000000001},"min":{"1min":0.221,"5min":0.193,"15min":0.193},"max":{"1min":0.497,"5min":0.755,"15min":0.755},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.35699999999999998,"15min":0.33900000000000002},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.58599999999999997,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.438}]},{"osd":1,"last update":"Thu Jan 25 16:05:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54400000000000004,"15min":0.53100000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54000000000000004,"15min":0.51500000000000001},"min":{"1min":0.39800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.624}]},{"osd":2,"last update":"Thu Jan 25 16:05:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.63800000000000001,"5min":0.877,"15min":0.877},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.626,"15min":0.57099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.41599999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.52500000000000002},"min":{"1min":0.40699999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.746}]},{"osd":5,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.63400000000000001,"15min":0.625},"min":{"1min":0.42699999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1,"5min":1,"15min":1},"last":0.55200000000000005}]},{"osd":6,"last update":"Thu Jan 25 16:04:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.51800000000000002,"15min":0.51500000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.373},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.498}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.61499999999999999,"15min":0.60299999999999998},"min":{"1min":0.5,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.47699999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69399999999999995}]},{"osd":8,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:05:10 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.56399999999999995}]},{"osd":10,"last update":"Thu Jan 25 16:05:32 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920816,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":443188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.438,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.35999999999999999,"15min":0.34899999999999998},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.58499999999999996,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.53400000000000003},"min":{"1min":0.39800000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.623,"5min":0.55500000000000005,"15min":0.53700000000000003},"min":{"1min":0.38200000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:05:36 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.57499999999999996,"15min":0.56000000000000005},"min":{"1min":0.41999999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.61399999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:04:52 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45600000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:05:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79600000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55700000000000005,"15min":0.56100000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40799999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:41 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.65400000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.70399999999999996,"15min":0.70199999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73799999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66200000000000003}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]}]},{"osd":5,"up_from":39,"seq":167503724588,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32688,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712464,"statfs":{"total":95995035648,"available":95961563136,"internally_reserved":0,"allocated":1548288,"data_stored":1034388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51900000000000002,"15min":0.51600000000000001},"min":{"1min":0.373,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.74299999999999999,"5min":0.751,"15min":0.751},"last":0.60799999999999998}]},{"osd":1,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57099999999999995,"15min":0.56599999999999995},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.55300000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.73299999999999998,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.64200000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40100000000000002},"min":{"1min":0.28100000000000003,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.27600000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.54900000000000004,"15min":0.55400000000000005},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.72799999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79900000000000004,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.751},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.753,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.83199999999999996}]},{"osd":6,"last update":"Thu Jan 25 16:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.627}]},{"osd":7,"last update":"Thu Jan 25 16:05:15 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65000000000000002,"15min":0.64900000000000002},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88400000000000001,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:05:39 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.38600000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.70399999999999996}]},{"osd":9,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.71799999999999997}]},{"osd":10,"last update":"Thu Jan 25 16:05:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-25T16:05:54.175 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-25T16:05:54.176 INFO:tasks.cephadm:Setting up client nodes... 2024-01-25T16:05:54.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-01-25T16:05:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:54 smithi152 ceph-mon[36015]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:54 smithi152 ceph-mon[36015]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:54 smithi152 ceph-mon[36015]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-01-25T16:05:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:54 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2272765719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:54 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3056421313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:54 smithi076 ceph-mon[35642]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:54 smithi076 ceph-mon[35642]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:54 smithi076 ceph-mon[35642]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-01-25T16:05:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:54 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2272765719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:54 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3056421313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:54.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:05:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:54 smithi033 ceph-mon[31417]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:54 smithi033 ceph-mon[31417]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:05:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:54 smithi033 ceph-mon[31417]: mgrmap e17: x(active, since 7m), standbys: y, z 2024-01-25T16:05:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:54 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2272765719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:05:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:54 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3056421313' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:05:55.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:55 smithi033 ceph-mon[31417]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:55.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:55 smithi033 ceph-mon[31417]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-01-25T16:05:55.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:55 smithi033 ceph-mon[31417]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:55 smithi152 ceph-mon[36015]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:55 smithi152 ceph-mon[36015]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-01-25T16:05:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:55 smithi152 ceph-mon[36015]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:55 smithi076 ceph-mon[35642]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:55 smithi076 ceph-mon[35642]: pgmap v289: 97 pgs: 97 active+clean; 581 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 4.7 KiB/s wr, 17 op/s 2024-01-25T16:05:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:55 smithi076 ceph-mon[35642]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:05:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 -- 172.21.15.33:0/1351188244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 msgr2=0x7f9920102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 --2- 172.21.15.33:0/1351188244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920102fc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9914009990 tx=0x7f9914009ca0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 -- 172.21.15.33:0/1351188244 shutdown_connections 2024-01-25T16:05:56.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 --2- 172.21.15.33:0/1351188244 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9920103500 0x7f9920109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 --2- 172.21.15.33:0/1351188244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99200ffb10 0x7f99200fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 --2- 172.21.15.33:0/1351188244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920102fc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 -- 172.21.15.33:0/1351188244 >> 172.21.15.33:0/1351188244 conn(0x7f99200fb480 msgr2=0x7f99200fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 -- 172.21.15.33:0/1351188244 shutdown_connections 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.102+0000 7f9926053700 1 -- 172.21.15.33:0/1351188244 wait complete. 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 Processor -- start 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 -- start start 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99200ffb10 0x7f9920197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 0x7f992019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019f310 con 0x7f9920102b40 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019f450 con 0x7f9920103500 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f9926053700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992019f5e0 con 0x7f99200ffb10 2024-01-25T16:05:56.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f991effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f991effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50860/0 (socket says 172.21.15.33:50860) 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.103+0000 7f991effd700 1 -- 172.21.15.33:0/4027147454 learned_addr learned my addr 172.21.15.33:0/4027147454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991f7fe700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99200ffb10 0x7f9920197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991ffff700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 0x7f992019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 -- 172.21.15.33:0/4027147454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99200ffb10 msgr2=0x7f9920197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99200ffb10 0x7f9920197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 -- 172.21.15.33:0/4027147454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 msgr2=0x7f992019ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 0x7f992019ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 -- 172.21.15.33:0/4027147454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140095f0 con 0x7f9920102b40 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991ffff700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 0x7f992019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991effd700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f99140059d0 tx=0x7f9914009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991401d070 con 0x7f9920102b40 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992019f860 con 0x7f9920102b40 2024-01-25T16:05:56.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9914004190 con 0x7f9920102b40 2024-01-25T16:05:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400f460 con 0x7f9920102b40 2024-01-25T16:05:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.104+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992019fd50 con 0x7f9920102b40 2024-01-25T16:05:56.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.105+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9920108d00 con 0x7f9920102b40 2024-01-25T16:05:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.106+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f9914003cd0 con 0x7f9920102b40 2024-01-25T16:05:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.107+0000 7f991cff9700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 0x7f99080a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.107+0000 7f991f7fe700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 0x7f99080a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.107+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99140cdfc0 con 0x7f9920102b40 2024-01-25T16:05:56.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.107+0000 7f991f7fe700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 0x7f99080a39f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9910006fd0 tx=0x7f9910008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.111+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f991405d600 con 0x7f9920102b40 2024-01-25T16:05:56.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.395+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f992004fcf0 con 0x7f9920102b40 2024-01-25T16:05:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.399+0000 7f991cff9700 1 -- 172.21.15.33:0/4027147454 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f9914097420 con 0x7f9920102b40 2024-01-25T16:05:56.402 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-01-25T16:05:56.402 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDkhrJl8SOlFxAA1kQdzSB7onI3OL5ovkyhzw== 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 msgr2=0x7f99080a39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 0x7f99080a39f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9910006fd0 tx=0x7f9910008040 comp rx=0 tx=0).stop 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 msgr2=0x7f9920197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f99140059d0 tx=0x7f9914009f70 comp rx=0 tx=0).stop 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 shutdown_connections 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99080a1530 0x7f99080a39f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9920103500 0x7f992019ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99200ffb10 0x7f9920197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 --2- 172.21.15.33:0/4027147454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9920102b40 0x7f9920197cc0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.402+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 >> 172.21.15.33:0/4027147454 conn(0x7f99200fb480 msgr2=0x7f9920105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.403+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 shutdown_connections 2024-01-25T16:05:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:56.403+0000 7f9926053700 1 -- 172.21.15.33:0/4027147454 wait complete. 2024-01-25T16:05:57.030 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T16:05:57.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-01-25T16:05:57.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-25T16:05:57.073 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-25T16:05:57.073 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-01-25T16:05:57.074 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph mgr dump --format=json 2024-01-25T16:05:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:57 smithi033 ceph-mon[31417]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-01-25T16:05:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:57 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-25T16:05:57.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:57 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-25T16:05:57.357 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:05:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:57 smithi076 ceph-mon[35642]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-01-25T16:05:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:57 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-25T16:05:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:57 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-25T16:05:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:57 smithi152 ceph-mon[36015]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 8.5 KiB/s wr, 19 op/s 2024-01-25T16:05:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:57 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-01-25T16:05:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:57 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4027147454' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-01-25T16:05:58.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 -- 172.21.15.33:0/1082592282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 msgr2=0x7fe268106ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1082592282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe268106ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe258009990 tx=0x7fe258009ca0 comp rx=0 tx=0).stop 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 -- 172.21.15.33:0/1082592282 shutdown_connections 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1082592282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2681021f0 0x7fe26810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1082592282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe268106ae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1082592282 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe268101830 0x7fe268101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 -- 172.21.15.33:0/1082592282 >> 172.21.15.33:0/1082592282 conn(0x7fe268078910 msgr2=0x7fe268078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 -- 172.21.15.33:0/1082592282 shutdown_connections 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.992+0000 7fe26cee1700 1 -- 172.21.15.33:0/1082592282 wait complete. 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 Processor -- start 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 -- start start 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe268101830 0x7fe268197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 0x7fe268198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819cc30 con 0x7fe268106700 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819cda0 con 0x7fe268101830 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe26cee1700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26819cf60 con 0x7fe2681021f0 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe266d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe266d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51102/0 (socket says 172.21.15.33:51102) 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.993+0000 7fe266d9d700 1 -- 172.21.15.33:0/1674286289 learned_addr learned my addr 172.21.15.33:0/1674286289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe26659c700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe268101830 0x7fe268197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe265d9b700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 0x7fe268198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 -- 172.21.15.33:0/1674286289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 msgr2=0x7fe268198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 0x7fe268198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:58.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 -- 172.21.15.33:0/1674286289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe268101830 msgr2=0x7fe268197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe268101830 0x7fe268197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 -- 172.21.15.33:0/1674286289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580095f0 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe265d9b700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 0x7fe268198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe266d9d700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b970 tx=0x7fe26000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26000c930 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe260014440 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26819d240 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2600075d0 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.994+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe268073be0 con 0x7fe268106700 2024-01-25T16:05:59.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.995+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268109d70 con 0x7fe268106700 2024-01-25T16:05:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.996+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fe2600145b0 con 0x7fe268106700 2024-01-25T16:05:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.997+0000 7fe2577fe700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 0x7fe2500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:05:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.997+0000 7fe26659c700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 0x7fe2500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:05:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.997+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe260006e40 con 0x7fe268106700 2024-01-25T16:05:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:58.997+0000 7fe26659c700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 0x7fe2500a39c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b5c0 tx=0x7fe258005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:05:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.001+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe2600955c0 con 0x7fe268106700 2024-01-25T16:05:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.260+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe2681990b0 con 0x7fe268106700 2024-01-25T16:05:59.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.265+0000 7fe2577fe700 1 -- 172.21.15.33:0/1674286289 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+259957 (secure 0 0 0) 0x7fe260098be0 con 0x7fe268106700 2024-01-25T16:05:59.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:05:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 msgr2=0x7fe2500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 0x7fe2500a39c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe25800b5c0 tx=0x7fe258005960 comp rx=0 tx=0).stop 2024-01-25T16:05:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 msgr2=0x7fe26819c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b970 tx=0x7fe26000bc80 comp rx=0 tx=0).stop 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 shutdown_connections 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe2500a1500 0x7fe2500a39c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe268106700 0x7fe26819c620 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe268101830 0x7fe268197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 --2- 172.21.15.33:0/1674286289 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2681021f0 0x7fe268198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 >> 172.21.15.33:0/1674286289 conn(0x7fe268078910 msgr2=0x7fe2680ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 shutdown_connections 2024-01-25T16:05:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:05:59.268+0000 7fe26cee1700 1 -- 172.21.15.33:0/1674286289 wait complete. 2024-01-25T16:05:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:05:59 smithi076 ceph-mon[35642]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-01-25T16:05:59.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:05:59 smithi152 ceph-mon[36015]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-01-25T16:05:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:05:59 smithi033 ceph-mon[31417]: pgmap v291: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 19 op/s 2024-01-25T16:05:59.930 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1063041969},{"type":"v1","addr":"172.21.15.33:6801","nonce":1063041969}]},"active_addr":"172.21.15.33:6801/1063041969","active_change":"2024-01-25T15:58:48.019001+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24110,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24112,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2511336685}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":69013124}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":441508626}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":679037113}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1925123535}]}]} 2024-01-25T16:05:59.936 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-01-25T16:05:59.937 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-01-25T16:05:59.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd dump --format=json 2024-01-25T16:06:00.179 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:00.413 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:00 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1674286289' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-25T16:06:00.488 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:00 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1674286289' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-25T16:06:00.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:00 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1674286289' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-01-25T16:06:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:01 smithi033 ceph-mon[31417]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-01-25T16:06:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:01 smithi076 ceph-mon[35642]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-01-25T16:06:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:01 smithi152 ceph-mon[36015]: pgmap v292: 97 pgs: 97 active+clean; 618 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 6.7 KiB/s wr, 17 op/s 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 -- 172.21.15.33:0/2758944900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241077c0 msgr2=0x7f8e24107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/2758944900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241077c0 0x7f8e24107ba0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c00b0a0 tx=0x7f8e1c00b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 -- 172.21.15.33:0/2758944900 shutdown_connections 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/2758944900 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e24102180 0x7f8e2410e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/2758944900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241077c0 0x7f8e24107ba0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/2758944900 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e241017c0 0x7f8e24101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 -- 172.21.15.33:0/2758944900 >> 172.21.15.33:0/2758944900 conn(0x7f8e240fd290 msgr2=0x7f8e240ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 -- 172.21.15.33:0/2758944900 shutdown_connections 2024-01-25T16:06:01.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.871+0000 7f8e2b92c700 1 -- 172.21.15.33:0/2758944900 wait complete. 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 Processor -- start 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 -- start start 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24102180 0x7f8e24074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 0x7f8e24074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241a61a0 con 0x7f8e241017c0 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241a6310 con 0x7f8e24102180 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e2b92c700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241a64d0 con 0x7f8e241077c0 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51120/0 (socket says 172.21.15.33:51120) 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 -- 172.21.15.33:0/3851740583 learned_addr learned my addr 172.21.15.33:0/3851740583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e28ec7700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24102180 0x7f8e24074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e29ec9700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 0x7f8e24074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 -- 172.21.15.33:0/3851740583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 msgr2=0x7f8e24074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:01.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 0x7f8e24074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 -- 172.21.15.33:0/3851740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24102180 msgr2=0x7f8e24074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24102180 0x7f8e24074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e296c8700 1 -- 172.21.15.33:0/3851740583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009d00 con 0x7f8e241017c0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.872+0000 7f8e29ec9700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 0x7f8e24074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e296c8700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0062a0 tx=0x7f8e1c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c00e040 con 0x7f8e241017c0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e1c004c60 con 0x7f8e241017c0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e241a6750 con 0x7f8e241017c0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c005280 con 0x7f8e241017c0 2024-01-25T16:06:01.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.873+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e241a6bc0 con 0x7f8e241017c0 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.874+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24109dc0 con 0x7f8e241017c0 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.875+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f8e1c0054a0 con 0x7f8e241017c0 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.875+0000 7f8e227fc700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 0x7f8e0c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.875+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8e1c0ce010 con 0x7f8e241017c0 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.875+0000 7f8e28ec7700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 0x7f8e0c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:01.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.876+0000 7f8e28ec7700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 0x7f8e0c0a38c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8e240ff930 tx=0x7f8e18009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:01.880+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8e1c097570 con 0x7f8e241017c0 2024-01-25T16:06:02.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.083+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e240684a0 con 0x7f8e241017c0 2024-01-25T16:06:02.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.085+0000 7f8e227fc700 1 -- 172.21.15.33:0/3851740583 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22024 (secure 0 0 0) 0x7f8e1c09ab90 con 0x7f8e241017c0 2024-01-25T16:06:02.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:06:02.089 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":86,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:05:47.781903+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:06:02.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 msgr2=0x7f8e0c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:02.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 0x7f8e0c0a38c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8e240ff930 tx=0x7f8e18009190 comp rx=0 tx=0).stop 2024-01-25T16:06:02.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 msgr2=0x7f8e24073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0062a0 tx=0x7f8e1c007870 comp rx=0 tx=0).stop 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 shutdown_connections 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8e0c0a1400 0x7f8e0c0a38c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e241077c0 0x7f8e24074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e241017c0 0x7f8e24073ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 --2- 172.21.15.33:0/3851740583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e24102180 0x7f8e24074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 >> 172.21.15.33:0/3851740583 conn(0x7f8e240fd290 msgr2=0x7f8e240fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 shutdown_connections 2024-01-25T16:06:02.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:02.087+0000 7f8e2b92c700 1 -- 172.21.15.33:0/3851740583 wait complete. 2024-01-25T16:06:02.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:02 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3851740583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:02.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:02 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3851740583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:02 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3851740583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:02.775 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-01-25T16:06:02.775 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd dump --format=json 2024-01-25T16:06:03.021 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:03.416 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:03 smithi033 ceph-mon[31417]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-01-25T16:06:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:03 smithi152 ceph-mon[36015]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-01-25T16:06:03.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:03 smithi076 ceph-mon[35642]: pgmap v293: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 17 op/s 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 -- 172.21.15.33:0/1254036286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 msgr2=0x7fccd810e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 --2- 172.21.15.33:0/1254036286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd810e690 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009990 tx=0x7fccd0009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 -- 172.21.15.33:0/1254036286 shutdown_connections 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 --2- 172.21.15.33:0/1254036286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd810e690 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 --2- 172.21.15.33:0/1254036286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 0x7fccd8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 --2- 172.21.15.33:0/1254036286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 0x7fccd8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 -- 172.21.15.33:0/1254036286 >> 172.21.15.33:0/1254036286 conn(0x7fccd80fd290 msgr2=0x7fccd80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 -- 172.21.15.33:0/1254036286 shutdown_connections 2024-01-25T16:06:04.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.735+0000 7fccdfcda700 1 -- 172.21.15.33:0/1254036286 wait complete. 2024-01-25T16:06:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 Processor -- start 2024-01-25T16:06:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 -- start start 2024-01-25T16:06:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 0x7fccd819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 0x7fccd8197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8198090 con 0x7fccd8102180 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81981d0 con 0x7fccd81017c0 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdfcda700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd8198390 con 0x7fccd81077c0 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51136/0 (socket says 172.21.15.33:51136) 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 -- 172.21.15.33:0/3246226188 learned_addr learned my addr 172.21.15.33:0/3246226188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdda76700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 0x7fccd819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccde277700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 0x7fccd8197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 -- 172.21.15.33:0/3246226188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 msgr2=0x7fccd8197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 0x7fccd8197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 -- 172.21.15.33:0/3246226188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 msgr2=0x7fccd819d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 0x7fccd819d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.736+0000 7fccdd275700 1 -- 172.21.15.33:0/3246226188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00095f0 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccdd275700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b530 tx=0x7fcccc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc011830 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcccc011e50 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd8198670 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc00f2e0 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.737+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd81a4910 con 0x7fccd8102180 2024-01-25T16:06:04.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.738+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd8109de0 con 0x7fccd8102180 2024-01-25T16:06:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.740+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fcccc0105a0 con 0x7fccd8102180 2024-01-25T16:06:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.740+0000 7fccd6ffd700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 0x7fccc00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.740+0000 7fccdda76700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 0x7fccc00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.740+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fcccc0cc960 con 0x7fccd8102180 2024-01-25T16:06:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.741+0000 7fccdda76700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 0x7fccc00a38c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccc80060b0 tx=0x7fccc800c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.744+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcccc095f40 con 0x7fccd8102180 2024-01-25T16:06:04.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.945+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccd80684a0 con 0x7fccd8102180 2024-01-25T16:06:04.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.947+0000 7fccd6ffd700 1 -- 172.21.15.33:0/3246226188 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22024 (secure 0 0 0) 0x7fcccc099560 con 0x7fccd8102180 2024-01-25T16:06:04.951 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:06:04.951 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":86,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:05:47.781903+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:06:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 msgr2=0x7fccc00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:04.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 0x7fccc00a38c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fccc80060b0 tx=0x7fccc800c040 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 msgr2=0x7fccd819ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00b530 tx=0x7fcccc00b840 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 shutdown_connections 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fccc00a1400 0x7fccc00a38c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fccd81077c0 0x7fccd8197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fccd81017c0 0x7fccd819d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 --2- 172.21.15.33:0/3246226188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccd8102180 0x7fccd819ddb0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 >> 172.21.15.33:0/3246226188 conn(0x7fccd80fd290 msgr2=0x7fccd80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 shutdown_connections 2024-01-25T16:06:04.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:04.949+0000 7fccdfcda700 1 -- 172.21.15.33:0/3246226188 wait complete. 2024-01-25T16:06:05.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:05 smithi033 ceph-mon[31417]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:05.253 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3246226188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:05 smithi152 ceph-mon[36015]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:05 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3246226188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:05.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-01-25T16:06:05.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-01-25T16:06:05.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-01-25T16:06:05.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-01-25T16:06:05.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-01-25T16:06:05.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-01-25T16:06:05.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-01-25T16:06:05.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-01-25T16:06:05.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-01-25T16:06:05.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-01-25T16:06:05.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-01-25T16:06:05.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-01-25T16:06:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:05 smithi076 ceph-mon[35642]: pgmap v294: 97 pgs: 97 active+clean; 624 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:05 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3246226188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:06:06.259 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.033 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.037 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.038 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.043 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.047 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.050 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:07.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:07 smithi033 ceph-mon[31417]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:07 smithi152 ceph-mon[36015]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:07 smithi076 ceph-mon[35642]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-01-25T16:06:07.874 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 -- 172.21.15.33:0/2588463906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc101830 msgr2=0x7f99cc101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 --2- 172.21.15.33:0/2588463906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc101830 0x7f99cc101cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0099c0 tx=0x7f99bc009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 -- 172.21.15.33:0/2588463906 shutdown_connections 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 --2- 172.21.15.33:0/2588463906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99cc1021f0 0x7f99cc10e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 --2- 172.21.15.33:0/2588463906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99cc106650 0x7f99cc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 --2- 172.21.15.33:0/2588463906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc101830 0x7f99cc101cb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 -- 172.21.15.33:0/2588463906 >> 172.21.15.33:0/2588463906 conn(0x7f99cc078910 msgr2=0x7f99cc078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 -- 172.21.15.33:0/2588463906 shutdown_connections 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.745+0000 7f99d334f700 1 -- 172.21.15.33:0/2588463906 wait complete. 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 Processor -- start 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 -- start start 2024-01-25T16:06:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99cc101830 0x7f99cc19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99cc1021f0 0x7f99cc19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 0x7f99cc197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc198110 con 0x7f99cc106650 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc198250 con 0x7f99cc101830 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d334f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc198410 con 0x7f99cc1021f0 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d18ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 0x7f99cc197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d18ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 0x7f99cc197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36674/0 (socket says 172.21.15.33:36674) 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d18ec700 1 -- 172.21.15.33:0/2468988296 learned_addr learned my addr 172.21.15.33:0/2468988296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d10eb700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99cc101830 0x7f99cc19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.746+0000 7f99d08ea700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99cc1021f0 0x7f99cc19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 -- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99cc1021f0 msgr2=0x7f99cc19de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99cc1021f0 0x7f99cc19de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 -- 172.21.15.33:0/2468988296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99cc101830 msgr2=0x7f99cc19d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99cc101830 0x7f99cc19d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0095f0 con 0x7f99cc106650 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d18ec700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 0x7f99cc197ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f99c4007ee0 tx=0x7f99c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4017070 con 0x7f99cc106650 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f99c400f040 con 0x7f99cc106650 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc1986f0 con 0x7f99cc106650 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4005580 con 0x7f99cc106650 2024-01-25T16:06:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.747+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc1a4990 con 0x7f99cc106650 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.748+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99cc102ec0 con 0x7f99cc106650 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.749+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f99c400db20 con 0x7f99cc106650 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.749+0000 7f99c27fc700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99b80a1480 0x7f99b80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99d10eb700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99b80a1480 0x7f99b80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f99c40cc880 con 0x7f99cc106650 2024-01-25T16:06:08.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99c27fc700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] conn(0x7f99b80aac60 0x7f99b80ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99b80ad730 con 0x7f99b80aac60 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99c40ccd20 con 0x7f99cc106650 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99d10eb700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99b80a1480 0x7f99b80a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99cc198bf0 tx=0x7f99c8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.750+0000 7f99d08ea700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] conn(0x7f99b80aac60 0x7f99b80ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.751+0000 7f99d08ea700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] conn(0x7f99b80aac60 0x7f99b80ad080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:08.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.751+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== osd.5 v2:172.21.15.152:6808/984769495 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f99b80ad730 con 0x7f99b80aac60 2024-01-25T16:06:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.768+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 --> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99cc111a60 con 0x7f99b80aac60 2024-01-25T16:06:08.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.768+0000 7f99c27fc700 1 -- 172.21.15.33:0/2468988296 <== osd.5 v2:172.21.15.152:6808/984769495 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f99cc111a60 con 0x7f99b80aac60 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] conn(0x7f99b80aac60 msgr2=0x7f99b80ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] conn(0x7f99b80aac60 0x7f99b80ad080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99b80a1480 msgr2=0x7f99b80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99b80a1480 0x7f99b80a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99cc198bf0 tx=0x7f99c8009190 comp rx=0 tx=0).stop 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 msgr2=0x7f99cc197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 --2- 172.21.15.33:0/2468988296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99cc106650 0x7f99cc197ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f99c4007ee0 tx=0x7f99c400d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d18ec700 1 -- 172.21.15.33:0/2468988296 reap_dead start 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 shutdown_connections 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 >> 172.21.15.33:0/2468988296 conn(0x7f99cc078910 msgr2=0x7f99cc0ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 shutdown_connections 2024-01-25T16:06:08.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:08.769+0000 7f99d334f700 1 -- 172.21.15.33:0/2468988296 wait complete. 2024-01-25T16:06:09.360 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:09 smithi033 ceph-mon[31417]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-01-25T16:06:09.387 INFO:teuthology.orchestra.run.smithi033.stdout:167503724592 2024-01-25T16:06:09.388 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-01-25T16:06:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:09 smithi152 ceph-mon[36015]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-01-25T16:06:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:09 smithi076 ceph-mon[35642]: pgmap v296: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-01-25T16:06:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:11 smithi033 ceph-mon[31417]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-01-25T16:06:11.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:11 smithi152 ceph-mon[36015]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-01-25T16:06:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:11 smithi076 ceph-mon[35642]: pgmap v297: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-01-25T16:06:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:13 smithi076 ceph-mon[35642]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-25T16:06:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:13 smithi033 ceph-mon[31417]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-25T16:06:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:13 smithi152 ceph-mon[36015]: pgmap v298: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-01-25T16:06:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:15 smithi076 ceph-mon[35642]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-01-25T16:06:15.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:15 smithi033 ceph-mon[31417]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-01-25T16:06:15.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:15 smithi152 ceph-mon[36015]: pgmap v299: 97 pgs: 97 active+clean; 634 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-01-25T16:06:15.755 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:17 smithi076 ceph-mon[35642]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:06:17.486 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:17 smithi033 ceph-mon[31417]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:06:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 -- 172.21.15.33:0/2827251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec406a840 msgr2=0x7f7ec410e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2827251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec406a840 0x7f7ec410e030 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4009990 tx=0x7f7eb4009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 -- 172.21.15.33:0/2827251145 shutdown_connections 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2827251145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ec410e570 0x7f7ec4110960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2827251145 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ec4069f20 0x7f7ec406a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2827251145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec406a840 0x7f7ec410e030 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 -- 172.21.15.33:0/2827251145 >> 172.21.15.33:0/2827251145 conn(0x7f7ec4077540 msgr2=0x7f7ec4077950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 -- 172.21.15.33:0/2827251145 shutdown_connections 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 -- 172.21.15.33:0/2827251145 wait complete. 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.591+0000 7f7ecae69700 1 Processor -- start 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 -- start start 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 0x7f7ec4199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ec406a840 0x7f7ec419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ec410e570 0x7f7ec419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419ed20 con 0x7f7ec4069f20 2024-01-25T16:06:17.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419ee90 con 0x7f7ec406a840 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ecae69700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec419f050 con 0x7f7ec410e570 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 0x7f7ec4199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 0x7f7ec4199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36692/0 (socket says 172.21.15.33:36692) 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 -- 172.21.15.33:0/2430098211 learned_addr learned my addr 172.21.15.33:0/2430098211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec9406700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ec410e570 0x7f7ec419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec3fff700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ec406a840 0x7f7ec419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 -- 172.21.15.33:0/2430098211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ec410e570 msgr2=0x7f7ec419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7ec410e570 0x7f7ec419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 -- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ec406a840 msgr2=0x7f7ec419a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7ec406a840 0x7f7ec419a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb40095f0 con 0x7f7ec4069f20 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec8c05700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 0x7f7ec4199e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0060b0 tx=0x7f7ebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.592+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00ddd0 con 0x7f7ec4069f20 2024-01-25T16:06:17.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.593+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec419f330 con 0x7f7ec4069f20 2024-01-25T16:06:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.593+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ebc00f040 con 0x7f7ec4069f20 2024-01-25T16:06:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.593+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc014690 con 0x7f7ec4069f20 2024-01-25T16:06:17.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.593+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec41a6b90 con 0x7f7ec4069f20 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.593+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ec4113cc0 con 0x7f7ec4069f20 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.595+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f7ebc015400 con 0x7f7ec4069f20 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.595+0000 7f7ec1ffb700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7eb00a1400 0x7f7eb00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.595+0000 7f7ec3fff700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7eb00a1400 0x7f7eb00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.595+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7ebc0ccc80 con 0x7f7ec4069f20 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec1ffb700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] conn(0x7f7eb00aabe0 0x7f7eb00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:17.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7eb00ad6b0 con 0x7f7eb00aabe0 2024-01-25T16:06:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ebc0cd120 con 0x7f7ec4069f20 2024-01-25T16:06:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec3fff700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7eb00a1400 0x7f7eb00a38c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005850 tx=0x7f7eb401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec9406700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] conn(0x7f7eb00aabe0 0x7f7eb00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.596+0000 7f7ec9406700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] conn(0x7f7eb00aabe0 0x7f7eb00ad000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.597+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== osd.7 v2:172.21.15.76:6816/610247607 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7eb00ad6b0 con 0x7f7eb00aabe0 2024-01-25T16:06:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.613+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 --> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ec4061e70 con 0x7f7eb00aabe0 2024-01-25T16:06:17.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.614+0000 7f7ec1ffb700 1 -- 172.21.15.33:0/2430098211 <== osd.7 v2:172.21.15.76:6816/610247607 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7ec4061e70 con 0x7f7eb00aabe0 2024-01-25T16:06:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.614+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] conn(0x7f7eb00aabe0 msgr2=0x7f7eb00ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.614+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] conn(0x7f7eb00aabe0 0x7f7eb00ad000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7eb00a1400 msgr2=0x7f7eb00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7eb00a1400 0x7f7eb00a38c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4005850 tx=0x7f7eb401a040 comp rx=0 tx=0).stop 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 msgr2=0x7f7ec4199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 --2- 172.21.15.33:0/2430098211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ec4069f20 0x7f7ec4199e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0060b0 tx=0x7f7ebc00d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ec9406700 1 -- 172.21.15.33:0/2430098211 reap_dead start 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 shutdown_connections 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 >> 172.21.15.33:0/2430098211 conn(0x7f7ec4077540 msgr2=0x7f7ec410fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 shutdown_connections 2024-01-25T16:06:17.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:17.615+0000 7f7ecae69700 1 -- 172.21.15.33:0/2430098211 wait complete. 2024-01-25T16:06:17.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:17 smithi152 ceph-mon[36015]: pgmap v300: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 -- 172.21.15.33:0/4241096356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1006d0 msgr2=0x7fc30c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 --2- 172.21.15.33:0/4241096356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1006d0 0x7fc30c100b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc300009990 tx=0x7fc300009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 -- 172.21.15.33:0/4241096356 shutdown_connections 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 --2- 172.21.15.33:0/4241096356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30c101090 0x7fc30c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 --2- 172.21.15.33:0/4241096356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc30c1066c0 0x7fc30c106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 --2- 172.21.15.33:0/4241096356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1006d0 0x7fc30c100b50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 -- 172.21.15.33:0/4241096356 >> 172.21.15.33:0/4241096356 conn(0x7fc30c077650 msgr2=0x7fc30c077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 -- 172.21.15.33:0/4241096356 shutdown_connections 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.519+0000 7fc3108df700 1 -- 172.21.15.33:0/4241096356 wait complete. 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 Processor -- start 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 -- start start 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc30c1006d0 0x7fc30c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30c101090 0x7fc30c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 0x7fc30c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c198170 con 0x7fc30c1066c0 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c1982b0 con 0x7fc30c101090 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.520+0000 7fc3108df700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30c198470 con 0x7fc30c1006d0 2024-01-25T16:06:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 0x7fc30c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 0x7fc30c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43490/0 (socket says 172.21.15.33:43490) 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 -- 172.21.15.33:0/3201294170 learned_addr learned my addr 172.21.15.33:0/3201294170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc309d9b700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30c101090 0x7fc30c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30a59c700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc30c1006d0 0x7fc30c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 -- 172.21.15.33:0/3201294170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc30c1006d0 msgr2=0x7fc30c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc30c1006d0 0x7fc30c19d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 -- 172.21.15.33:0/3201294170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30c101090 msgr2=0x7fc30c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc30c101090 0x7fc30c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3000095f0 con 0x7fc30c1066c0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc30ad9d700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 0x7fc30c197b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc304007ee0 tx=0x7fc30400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304017070 con 0x7fc30c1066c0 2024-01-25T16:06:18.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30c198750 con 0x7fc30c1066c0 2024-01-25T16:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc30400f040 con 0x7fc30c1066c0 2024-01-25T16:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c1a49a0 con 0x7fc30c1066c0 2024-01-25T16:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.521+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304005740 con 0x7fc30c1066c0 2024-01-25T16:06:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.522+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc2ec001030 con 0x7fc30c1066c0 2024-01-25T16:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.523+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fc3040058a0 con 0x7fc30c1066c0 2024-01-25T16:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc2fb7fe700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2f40a1400 0x7fc2f40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc30a59c700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2f40a1400 0x7fc2f40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3040ccce0 con 0x7fc30c1066c0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc2fb7fe700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] conn(0x7fc2f40aabe0 0x7fc2f40ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc2f40ad6b0 con 0x7fc2f40aabe0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3040cd180 con 0x7fc30c1066c0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc309d9b700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] conn(0x7fc2f40aabe0 0x7fc2f40ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.524+0000 7fc30a59c700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2f40a1400 0x7fc2f40a38c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.525+0000 7fc309d9b700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] conn(0x7fc2f40aabe0 0x7fc2f40ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.525+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== osd.3 v2:172.21.15.33:6810/3993087703 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc2f40ad6b0 con 0x7fc2f40aabe0 2024-01-25T16:06:18.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.542+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 --> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc2ec002dc0 con 0x7fc2f40aabe0 2024-01-25T16:06:18.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.542+0000 7fc2fb7fe700 1 -- 172.21.15.33:0/3201294170 <== osd.3 v2:172.21.15.33:6810/3993087703 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc2ec002dc0 con 0x7fc2f40aabe0 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] conn(0x7fc2f40aabe0 msgr2=0x7fc2f40ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] conn(0x7fc2f40aabe0 0x7fc2f40ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2f40a1400 msgr2=0x7fc2f40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2f40a1400 0x7fc2f40a38c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc005fd0 tx=0x7fc2fc005db0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 msgr2=0x7fc30c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 --2- 172.21.15.33:0/3201294170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc30c1066c0 0x7fc30c197b10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc304007ee0 tx=0x7fc30400d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc30ad9d700 1 -- 172.21.15.33:0/3201294170 reap_dead start 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 shutdown_connections 2024-01-25T16:06:18.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 >> 172.21.15.33:0/3201294170 conn(0x7fc30c077650 msgr2=0x7fc30c06a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:18.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 shutdown_connections 2024-01-25T16:06:18.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:18.543+0000 7fc3108df700 1 -- 172.21.15.33:0/3201294170 wait complete. 2024-01-25T16:06:19.508 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:19 smithi033 ceph-mon[31417]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:19 smithi152 ceph-mon[36015]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:19.645 INFO:teuthology.orchestra.run.smithi033.stdout:223338299430 2024-01-25T16:06:19.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-01-25T16:06:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:19 smithi076 ceph-mon[35642]: pgmap v301: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 -- 172.21.15.33:0/1780703980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e81010d0 msgr2=0x7f49e810d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 --2- 172.21.15.33:0/1780703980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e81010d0 0x7f49e810d550 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b0a0 tx=0x7f49e000b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 -- 172.21.15.33:0/1780703980 shutdown_connections 2024-01-25T16:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 --2- 172.21.15.33:0/1780703980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e81010d0 0x7f49e810d550 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 --2- 172.21.15.33:0/1780703980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49e8106680 0x7f49e8106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 --2- 172.21.15.33:0/1780703980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49e8100710 0x7f49e8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 -- 172.21.15.33:0/1780703980 >> 172.21.15.33:0/1780703980 conn(0x7f49e8076730 msgr2=0x7f49e8076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 -- 172.21.15.33:0/1780703980 shutdown_connections 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.613+0000 7f49ee909700 1 -- 172.21.15.33:0/1780703980 wait complete. 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 Processor -- start 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 -- start start 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 0x7f49e8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49e81010d0 0x7f49e819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49e8106680 0x7f49e819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819edc0 con 0x7f49e8100710 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819ef30 con 0x7f49e81010d0 2024-01-25T16:06:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49ee909700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819f0f0 con 0x7f49e8106680 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 0x7f49e8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49e7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 0x7f49e8199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43518/0 (socket says 172.21.15.33:43518) 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.614+0000 7f49e7fff700 1 -- 172.21.15.33:0/590691241 learned_addr learned my addr 172.21.15.33:0/590691241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49ecea6700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49e8106680 0x7f49e819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e77fe700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49e81010d0 0x7f49e819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 -- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49e8106680 msgr2=0x7f49e819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49e8106680 0x7f49e819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 -- 172.21.15.33:0/590691241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49e81010d0 msgr2=0x7f49e819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49e81010d0 0x7f49e819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0009d00 con 0x7f49e8100710 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e7fff700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 0x7f49e8199ee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e9a0 tx=0x7f49d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8009790 con 0x7f49e8100710 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49d8010430 con 0x7f49e8100710 2024-01-25T16:06:19.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8018680 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e819f3d0 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.615+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e81a6c30 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.616+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49e8110940 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.618+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f49d80105a0 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e57fa700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f49d00a1480 0x7f49d00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f49d8014070 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e77fe700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f49d00a1480 0x7f49d00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e57fa700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] conn(0x7f49d00aac50 0x7f49d00ad070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49d00ad720 con 0x7f49d00aac50 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.619+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d80cda10 con 0x7f49e8100710 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.620+0000 7f49e77fe700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f49d00a1480 0x7f49d00a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f49e8076fd0 tx=0x7f49dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.620+0000 7f49ecea6700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] conn(0x7f49d00aac50 0x7f49d00ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.620+0000 7f49ecea6700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] conn(0x7f49d00aac50 0x7f49d00ad070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:19.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.621+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== osd.2 v2:172.21.15.152:6800/1513420968 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f49d00ad720 con 0x7f49d00aac50 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.638+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 --> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49e8061ef0 con 0x7f49d00aac50 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.638+0000 7f49e57fa700 1 -- 172.21.15.33:0/590691241 <== osd.2 v2:172.21.15.152:6800/1513420968 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f49e8061ef0 con 0x7f49d00aac50 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] conn(0x7f49d00aac50 msgr2=0x7f49d00ad070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] conn(0x7f49d00aac50 0x7f49d00ad070 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f49d00a1480 msgr2=0x7f49d00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f49d00a1480 0x7f49d00a3940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f49e8076fd0 tx=0x7f49dc0073d0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 msgr2=0x7f49e8199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 --2- 172.21.15.33:0/590691241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49e8100710 0x7f49e8199ee0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e9a0 tx=0x7f49d800ecb0 comp rx=0 tx=0).stop 2024-01-25T16:06:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ecea6700 1 -- 172.21.15.33:0/590691241 reap_dead start 2024-01-25T16:06:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 shutdown_connections 2024-01-25T16:06:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 >> 172.21.15.33:0/590691241 conn(0x7f49e8076730 msgr2=0x7f49e80fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 shutdown_connections 2024-01-25T16:06:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.639+0000 7f49ee909700 1 -- 172.21.15.33:0/590691241 wait complete. 2024-01-25T16:06:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 -- 172.21.15.33:0/3420380381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8100790 msgr2=0x7fb9d810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 --2- 172.21.15.33:0/3420380381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8100790 0x7fb9d810e6f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b0a0 tx=0x7fb9d000b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 -- 172.21.15.33:0/3420380381 shutdown_connections 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 --2- 172.21.15.33:0/3420380381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8100790 0x7fb9d810e6f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 --2- 172.21.15.33:0/3420380381 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9d8106850 0x7fb9d8106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 --2- 172.21.15.33:0/3420380381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d80ffdd0 0x7fb9d8100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 -- 172.21.15.33:0/3420380381 >> 172.21.15.33:0/3420380381 conn(0x7fb9d8076ce0 msgr2=0x7fb9d80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 -- 172.21.15.33:0/3420380381 shutdown_connections 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.522+0000 7fb9dd083700 1 -- 172.21.15.33:0/3420380381 wait complete. 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 Processor -- start 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 -- start start 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9d80ffdd0 0x7fb9d8100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d8100790 0x7fb9d81014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 0x7fb9d8101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81a6370 con 0x7fb9d8106850 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81a64e0 con 0x7fb9d8100790 2024-01-25T16:06:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9dd083700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d81a66a0 con 0x7fb9d80ffdd0 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.523+0000 7fb9d759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 0x7fb9d8101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 0x7fb9d8101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43502/0 (socket says 172.21.15.33:43502) 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 -- 172.21.15.33:0/1032930351 learned_addr learned my addr 172.21.15.33:0/1032930351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d6d9d700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9d80ffdd0 0x7fb9d8100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d659c700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d8100790 0x7fb9d81014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 -- 172.21.15.33:0/1032930351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9d80ffdd0 msgr2=0x7fb9d8100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9d80ffdd0 0x7fb9d8100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 -- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d8100790 msgr2=0x7fb9d81014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d8100790 0x7fb9d81014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009d00 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d659c700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9d8100790 0x7fb9d81014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9d759e700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 0x7fb9d8101a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b830 tx=0x7fb9d00078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000e040 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9d0004e60 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d81a6920 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0005440 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.524+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d81a6de0 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.525+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb9d8111b40 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fb9d001bb30 con 0x7fb9d8106850 2024-01-25T16:06:20.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9c00a1480 0x7fb9c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9d6d9d700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9c00a1480 0x7fb9c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb9d00ce6b0 con 0x7fb9d8106850 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] conn(0x7fb9c00aac60 0x7fb9c00ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb9c00ad730 con 0x7fb9c00aac60 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.527+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d00ceb50 con 0x7fb9d8106850 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.528+0000 7fb9d6d9d700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9c00a1480 0x7fb9c00a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8005fd0 tx=0x7fb9c800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.528+0000 7fb9d659c700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] conn(0x7fb9c00aac60 0x7fb9c00ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.528+0000 7fb9d659c700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] conn(0x7fb9c00aac60 0x7fb9c00ad080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.529+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== osd.4 v2:172.21.15.76:6808/1997772054 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb9c00ad730 con 0x7fb9c00aac60 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.546+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 --> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb9d8061eb0 con 0x7fb9c00aac60 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.546+0000 7fb9bffff700 1 -- 172.21.15.33:0/1032930351 <== osd.4 v2:172.21.15.76:6808/1997772054 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb9d8061eb0 con 0x7fb9c00aac60 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] conn(0x7fb9c00aac60 msgr2=0x7fb9c00ad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] conn(0x7fb9c00aac60 0x7fb9c00ad080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9c00a1480 msgr2=0x7fb9c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb9c00a1480 0x7fb9c00a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8005fd0 tx=0x7fb9c800a560 comp rx=0 tx=0).stop 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 msgr2=0x7fb9d8101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 --2- 172.21.15.33:0/1032930351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9d8106850 0x7fb9d8101a30 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000b830 tx=0x7fb9d00078a0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9d759e700 1 -- 172.21.15.33:0/1032930351 reap_dead start 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 shutdown_connections 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 >> 172.21.15.33:0/1032930351 conn(0x7fb9d8076ce0 msgr2=0x7fb9d8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 shutdown_connections 2024-01-25T16:06:20.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:19.547+0000 7fb9dd083700 1 -- 172.21.15.33:0/1032930351 wait complete. 2024-01-25T16:06:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 -- 172.21.15.33:0/3307263910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec101090 msgr2=0x7f52ec10d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/3307263910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec101090 0x7f52ec10d520 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b0a0 tx=0x7f52e400b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 -- 172.21.15.33:0/3307263910 shutdown_connections 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/3307263910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec101090 0x7f52ec10d520 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/3307263910 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec1066c0 0x7f52ec106aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/3307263910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52ec1006d0 0x7f52ec100b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 -- 172.21.15.33:0/3307263910 >> 172.21.15.33:0/3307263910 conn(0x7f52ec077650 msgr2=0x7f52ec077a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 -- 172.21.15.33:0/3307263910 shutdown_connections 2024-01-25T16:06:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.658+0000 7f52f2ae6700 1 -- 172.21.15.33:0/3307263910 wait complete. 2024-01-25T16:06:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.659+0000 7f52f2ae6700 1 Processor -- start 2024-01-25T16:06:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.659+0000 7f52f2ae6700 1 -- start start 2024-01-25T16:06:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec1006d0 0x7f52ec102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52ec101090 0x7f52ec1027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 0x7f52ec102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a60b0 con 0x7f52ec1066c0 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f1083700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 0x7f52ec102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f1083700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 0x7f52ec102d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43530/0 (socket says 172.21.15.33:43530) 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f1083700 1 -- 172.21.15.33:0/2073342946 learned_addr learned my addr 172.21.15.33:0/2073342946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f0882700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec1006d0 0x7f52ec102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52ebfff700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52ec101090 0x7f52ec1027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a6220 con 0x7f52ec101090 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.660+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec1a63e0 con 0x7f52ec1006d0 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 -- 172.21.15.33:0/2073342946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec1006d0 msgr2=0x7f52ec102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f52ec1006d0 0x7f52ec102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 -- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52ec101090 msgr2=0x7f52ec1027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52ec101090 0x7f52ec1027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009d00 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f1083700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 0x7f52ec102d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b070 tx=0x7f52e4007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e400e040 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52e4009440 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1a6660 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e401b450 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.661+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1a6bb0 con 0x7f52ec1066c0 2024-01-25T16:06:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.662+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52ec110960 con 0x7f52ec1066c0 2024-01-25T16:06:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.664+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f52e401d480 con 0x7f52ec1066c0 2024-01-25T16:06:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.665+0000 7f52e9ffb700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f52dc0a1500 0x7f52dc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.665+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52e40cead0 con 0x7f52ec1066c0 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.665+0000 7f52e9ffb700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] conn(0x7f52dc0aace0 0x7f52dc0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.665+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52dc0ad7b0 con 0x7f52dc0aace0 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.665+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e40cee80 con 0x7f52ec1066c0 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.666+0000 7f52f0882700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f52dc0a1500 0x7f52dc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.666+0000 7f52ebfff700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] conn(0x7f52dc0aace0 0x7f52dc0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.666+0000 7f52f0882700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f52dc0a1500 0x7f52dc0a39c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005950 tx=0x7f52e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.666+0000 7f52ebfff700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] conn(0x7f52dc0aace0 0x7f52dc0ad100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:20.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.668+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== osd.1 v2:172.21.15.76:6800/2892864635 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f52dc0ad7b0 con 0x7f52dc0aace0 2024-01-25T16:06:20.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.685+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 --> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52ec104610 con 0x7f52dc0aace0 2024-01-25T16:06:20.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.685+0000 7f52e9ffb700 1 -- 172.21.15.33:0/2073342946 <== osd.1 v2:172.21.15.76:6800/2892864635 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52ec104610 con 0x7f52dc0aace0 2024-01-25T16:06:20.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] conn(0x7f52dc0aace0 msgr2=0x7f52dc0ad100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] conn(0x7f52dc0aace0 0x7f52dc0ad100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f52dc0a1500 msgr2=0x7f52dc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f52dc0a1500 0x7f52dc0a39c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005950 tx=0x7f52e000b410 comp rx=0 tx=0).stop 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 msgr2=0x7f52ec102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 --2- 172.21.15.33:0/2073342946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52ec1066c0 0x7f52ec102d00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b070 tx=0x7f52e4007870 comp rx=0 tx=0).stop 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f1083700 1 -- 172.21.15.33:0/2073342946 reap_dead start 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 shutdown_connections 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 >> 172.21.15.33:0/2073342946 conn(0x7f52ec077650 msgr2=0x7f52ec0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 shutdown_connections 2024-01-25T16:06:20.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:20.686+0000 7f52f2ae6700 1 -- 172.21.15.33:0/2073342946 wait complete. 2024-01-25T16:06:21.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:21 smithi033 ceph-mon[31417]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:21.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:21 smithi152 ceph-mon[36015]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:21 smithi076 ceph-mon[35642]: pgmap v302: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:21.901 INFO:teuthology.orchestra.run.smithi033.stdout:107374182460 2024-01-25T16:06:21.901 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-01-25T16:06:23.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:23 smithi076 ceph-mon[35642]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-25T16:06:23.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:23 smithi033 ceph-mon[31417]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-25T16:06:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:23 smithi152 ceph-mon[36015]: pgmap v303: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s 2024-01-25T16:06:23.880 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:23.902 INFO:teuthology.orchestra.run.smithi033.stdout:77309411393 2024-01-25T16:06:23.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-01-25T16:06:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 -- 172.21.15.33:0/3130238285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 msgr2=0x7fbb98101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3130238285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb98101cb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c0099c0 tx=0x7fbb8c009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 -- 172.21.15.33:0/3130238285 shutdown_connections 2024-01-25T16:06:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3130238285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981021f0 0x7fbb9810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3130238285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb98106650 0x7fbb98106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3130238285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb98101cb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 -- 172.21.15.33:0/3130238285 >> 172.21.15.33:0/3130238285 conn(0x7fbb98078910 msgr2=0x7fbb98078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 -- 172.21.15.33:0/3130238285 shutdown_connections 2024-01-25T16:06:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.366+0000 7fbba02a8700 1 -- 172.21.15.33:0/3130238285 wait complete. 2024-01-25T16:06:24.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.367+0000 7fbba02a8700 1 Processor -- start 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.367+0000 7fbba02a8700 1 -- start start 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb9819d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb981021f0 0x7fbb9819de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb98106650 0x7fbb98197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98198100 con 0x7fbb98101830 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98198240 con 0x7fbb98106650 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbba02a8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb98198400 con 0x7fbb981021f0 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb9819d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb9819d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43562/0 (socket says 172.21.15.33:43562) 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 -- 172.21.15.33:0/3902813985 learned_addr learned my addr 172.21.15.33:0/3902813985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:24.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9d843700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb981021f0 0x7fbb9819de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 -- 172.21.15.33:0/3902813985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb981021f0 msgr2=0x7fbb9819de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb981021f0 0x7fbb9819de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 -- 172.21.15.33:0/3902813985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb98106650 msgr2=0x7fbb98197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb98106650 0x7fbb98197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0095f0 con 0x7fbb98101830 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.368+0000 7fbb9e044700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb9819d8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007f10 tx=0x7fbb8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.369+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800de90 con 0x7fbb98101830 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.369+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb981986e0 con 0x7fbb98101830 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.369+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb8800f040 con 0x7fbb98101830 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.369+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88004540 con 0x7fbb98101830 2024-01-25T16:06:24.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.369+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb981a4980 con 0x7fbb98101830 2024-01-25T16:06:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.370+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb98102ec0 con 0x7fbb98101830 2024-01-25T16:06:24.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.372+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fbb88004ad0 con 0x7fbb98101830 2024-01-25T16:06:24.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.372+0000 7fbb977fe700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a1480 0x7fbb800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.372+0000 7fbb9d843700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a1480 0x7fbb800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.372+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbb880cd200 con 0x7fbb98101830 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb977fe700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] conn(0x7fbb800aaaf0 0x7fbb800acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb800ad5c0 con 0x7fbb800aaaf0 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb880cd6a0 con 0x7fbb98101830 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb9e845700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] conn(0x7fbb800aaaf0 0x7fbb800acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb9d843700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a1480 0x7fbb800a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009990 tx=0x7fbb8c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.373+0000 7fbb9e845700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] conn(0x7fbb800aaaf0 0x7fbb800acf10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.374+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== osd.6 v2:172.21.15.33:6818/2008861778 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbb800ad5c0 con 0x7fbb800aaaf0 2024-01-25T16:06:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.390+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 --> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb98111a30 con 0x7fbb800aaaf0 2024-01-25T16:06:24.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbb977fe700 1 -- 172.21.15.33:0/3902813985 <== osd.6 v2:172.21.15.33:6818/2008861778 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbb98111a30 con 0x7fbb800aaaf0 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] conn(0x7fbb800aaaf0 msgr2=0x7fbb800acf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] conn(0x7fbb800aaaf0 0x7fbb800acf10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a1480 msgr2=0x7fbb800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a1480 0x7fbb800a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c009990 tx=0x7fbb8c003680 comp rx=0 tx=0).stop 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 msgr2=0x7fbb9819d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 --2- 172.21.15.33:0/3902813985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98101830 0x7fbb9819d8e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbb88007f10 tx=0x7fbb8800d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbb9e845700 1 -- 172.21.15.33:0/3902813985 reap_dead start 2024-01-25T16:06:24.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 shutdown_connections 2024-01-25T16:06:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 >> 172.21.15.33:0/3902813985 conn(0x7fbb98078910 msgr2=0x7fbb980ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 shutdown_connections 2024-01-25T16:06:24.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.391+0000 7fbba02a8700 1 -- 172.21.15.33:0/3902813985 wait complete. 2024-01-25T16:06:24.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 -- 172.21.15.33:0/2535430695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201066c0 msgr2=0x7f8f20106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2535430695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201066c0 0x7f8f20106aa0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009990 tx=0x7f8f1c009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 -- 172.21.15.33:0/2535430695 shutdown_connections 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2535430695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f201010d0 0x7f8f2010d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2535430695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201066c0 0x7f8f20106aa0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2535430695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f20100710 0x7f8f20100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 -- 172.21.15.33:0/2535430695 >> 172.21.15.33:0/2535430695 conn(0x7f8f20076720 msgr2=0x7f8f20076b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 -- 172.21.15.33:0/2535430695 shutdown_connections 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.596+0000 7f8f279ad700 1 -- 172.21.15.33:0/2535430695 wait complete. 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 Processor -- start 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 -- start start 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20100710 0x7f8f20198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 0x7f8f201990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f201066c0 0x7f8f201a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f201999a0 con 0x7f8f201010d0 2024-01-25T16:06:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f201a4160 con 0x7f8f20100710 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f279ad700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f201a42a0 con 0x7f8f201066c0 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f24f48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 0x7f8f201990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f24f48700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 0x7f8f201990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43576/0 (socket says 172.21.15.33:43576) 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.597+0000 7f8f24f48700 1 -- 172.21.15.33:0/2001102695 learned_addr learned my addr 172.21.15.33:0/2001102695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 -- 172.21.15.33:0/2001102695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f201066c0 msgr2=0x7f8f201a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8f201066c0 0x7f8f201a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f25749700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20100710 0x7f8f20198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 -- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20100710 msgr2=0x7f8f20198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f20100710 0x7f8f20198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1c0095f0 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f24f48700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 0x7f8f201990e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000d980 tx=0x7f8f1000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000bb60 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f10007640 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f201a4580 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f10004e90 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.598+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f201a4ad0 con 0x7f8f201010d0 2024-01-25T16:06:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.599+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f20110990 con 0x7f8f201010d0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.600+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f8f1000bcc0 con 0x7f8f201010d0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.600+0000 7f8f167fc700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8f0c0a1480 0x7f8f0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f25749700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8f0c0a1480 0x7f8f0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8f100cd020 con 0x7f8f201010d0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f167fc700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] conn(0x7f8f0c0aaaf0 0x7f8f0c0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f0c0ad5c0 con 0x7f8f0c0aaaf0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f100cd4c0 con 0x7f8f201010d0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f25749700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8f0c0a1480 0x7f8f0c0a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0095c0 tx=0x7f8f1c009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f25f4a700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] conn(0x7f8f0c0aaaf0 0x7f8f0c0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.601+0000 7f8f25f4a700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] conn(0x7f8f0c0aaaf0 0x7f8f0c0acf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.602+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== osd.10 v2:172.21.15.76:6824/772498776 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8f0c0ad5c0 con 0x7f8f0c0aaaf0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.619+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 --> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f20061e70 con 0x7f8f0c0aaaf0 2024-01-25T16:06:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.619+0000 7f8f167fc700 1 -- 172.21.15.33:0/2001102695 <== osd.10 v2:172.21.15.76:6824/772498776 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8f20061e70 con 0x7f8f0c0aaaf0 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] conn(0x7f8f0c0aaaf0 msgr2=0x7f8f0c0acf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] conn(0x7f8f0c0aaaf0 0x7f8f0c0acf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8f0c0a1480 msgr2=0x7f8f0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8f0c0a1480 0x7f8f0c0a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0095c0 tx=0x7f8f1c009530 comp rx=0 tx=0).stop 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 msgr2=0x7f8f201990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 --2- 172.21.15.33:0/2001102695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f201010d0 0x7f8f201990e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8f1000d980 tx=0x7f8f1000dd40 comp rx=0 tx=0).stop 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f25f4a700 1 -- 172.21.15.33:0/2001102695 reap_dead start 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 shutdown_connections 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 >> 172.21.15.33:0/2001102695 conn(0x7f8f20076720 msgr2=0x7f8f200ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 shutdown_connections 2024-01-25T16:06:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:24.620+0000 7f8f279ad700 1 -- 172.21.15.33:0/2001102695 wait complete. 2024-01-25T16:06:25.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:25 smithi033 ceph-mon[31417]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:25 smithi076 ceph-mon[35642]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:25.556 INFO:teuthology.orchestra.run.smithi033.stdout:55834574917 2024-01-25T16:06:25.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-01-25T16:06:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:25 smithi152 ceph-mon[36015]: pgmap v304: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-01-25T16:06:25.982 INFO:teuthology.orchestra.run.smithi033.stdout:141733920822 2024-01-25T16:06:25.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-01-25T16:06:26.661 INFO:teuthology.orchestra.run.smithi033.stdout:313532612633 2024-01-25T16:06:26.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-01-25T16:06:27.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:27 smithi033 ceph-mon[31417]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:27 smithi076 ceph-mon[35642]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:27.626 INFO:teuthology.orchestra.run.smithi033.stdout:201863462958 2024-01-25T16:06:27.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-01-25T16:06:27.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:27 smithi152 ceph-mon[36015]: pgmap v305: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 6 op/s 2024-01-25T16:06:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 -- 172.21.15.33:0/2920590339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0107920 msgr2=0x7f42f0107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 --2- 172.21.15.33:0/2920590339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0107920 0x7f42f0107d00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009990 tx=0x7f42ec009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 -- 172.21.15.33:0/2920590339 shutdown_connections 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 --2- 172.21.15.33:0/2920590339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f01002c0 0x7f42f010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 --2- 172.21.15.33:0/2920590339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f0107920 0x7f42f0107d00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 --2- 172.21.15.33:0/2920590339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f00ff900 0x7f42f00ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.077+0000 7f42f8777700 1 -- 172.21.15.33:0/2920590339 >> 172.21.15.33:0/2920590339 conn(0x7f42f00768b0 msgr2=0x7f42f00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- 172.21.15.33:0/2920590339 shutdown_connections 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- 172.21.15.33:0/2920590339 wait complete. 2024-01-25T16:06:29.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 Processor -- start 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- start start 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 0x7f42f0101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f01002c0 0x7f42f01018b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f0107920 0x7f42f0101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01a5ef0 con 0x7f42f00ff900 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01a6060 con 0x7f42f0107920 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f8777700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01a6220 con 0x7f42f01002c0 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f6513700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 0x7f42f0101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f6513700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 0x7f42f0101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34292/0 (socket says 172.21.15.33:34292) 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.078+0000 7f42f6513700 1 -- 172.21.15.33:0/3888145988 learned_addr learned my addr 172.21.15.33:0/3888145988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6d14700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f0107920 0x7f42f0101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f5d12700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f01002c0 0x7f42f01018b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 -- 172.21.15.33:0/3888145988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f01002c0 msgr2=0x7f42f01018b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42f01002c0 0x7f42f01018b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 -- 172.21.15.33:0/3888145988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f0107920 msgr2=0x7f42f0101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f0107920 0x7f42f0101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0095f0 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6d14700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42f0107920 0x7f42f0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f6513700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 0x7f42f0101370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009fd0 tx=0x7f42ec00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec01d070 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42ec005510 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f01a64a0 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec004bd0 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.079+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f01a6940 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.080+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42f0100cd0 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.081+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f42ec004d30 con 0x7f42f00ff900 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42e77fe700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f42dc0a1400 0x7f42dc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:29.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42f5d12700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f42dc0a1400 0x7f42dc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42ec0ce020 con 0x7f42f00ff900 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42e77fe700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] conn(0x7f42dc0aabe0 0x7f42dc0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f42dc0ad6b0 con 0x7f42dc0aabe0 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ec0ce4c0 con 0x7f42f00ff900 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42f6d14700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] conn(0x7f42dc0aabe0 0x7f42dc0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42f5d12700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f42dc0a1400 0x7f42dc0a38c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005950 tx=0x7f42e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:29.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.082+0000 7f42f6d14700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] conn(0x7f42dc0aabe0 0x7f42dc0ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:29.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.083+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== osd.9 v2:172.21.15.33:6826/877151460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f42dc0ad6b0 con 0x7f42dc0aabe0 2024-01-25T16:06:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.099+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 --> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f42f01119b0 con 0x7f42dc0aabe0 2024-01-25T16:06:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42e77fe700 1 -- 172.21.15.33:0/3888145988 <== osd.9 v2:172.21.15.33:6826/877151460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f42f01119b0 con 0x7f42dc0aabe0 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] conn(0x7f42dc0aabe0 msgr2=0x7f42dc0ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] conn(0x7f42dc0aabe0 0x7f42dc0ad000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f42dc0a1400 msgr2=0x7f42dc0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f42dc0a1400 0x7f42dc0a38c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f42e0005950 tx=0x7f42e00058e0 comp rx=0 tx=0).stop 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 msgr2=0x7f42f0101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 --2- 172.21.15.33:0/3888145988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42f00ff900 0x7f42f0101370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009fd0 tx=0x7f42ec00f720 comp rx=0 tx=0).stop 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f6d14700 1 -- 172.21.15.33:0/3888145988 reap_dead start 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 shutdown_connections 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 >> 172.21.15.33:0/3888145988 conn(0x7f42f00768b0 msgr2=0x7f42f00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 shutdown_connections 2024-01-25T16:06:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.100+0000 7f42f8777700 1 -- 172.21.15.33:0/3888145988 wait complete. 2024-01-25T16:06:29.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:29 smithi033 ceph-mon[31417]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:29.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:29 smithi152 ceph-mon[36015]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:29.671 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:29 smithi076 ceph-mon[35642]: pgmap v306: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 -- 172.21.15.33:0/3393522952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204101830 msgr2=0x7fd204101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 --2- 172.21.15.33:0/3393522952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204101830 0x7fd204101cb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd1f40099c0 tx=0x7fd1f4009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 -- 172.21.15.33:0/3393522952 shutdown_connections 2024-01-25T16:06:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 --2- 172.21.15.33:0/3393522952 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2041021f0 0x7fd20410e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 --2- 172.21.15.33:0/3393522952 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd204106650 0x7fd204106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 --2- 172.21.15.33:0/3393522952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd204101830 0x7fd204101cb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 -- 172.21.15.33:0/3393522952 >> 172.21.15.33:0/3393522952 conn(0x7fd204078910 msgr2=0x7fd204078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 -- 172.21.15.33:0/3393522952 shutdown_connections 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.310+0000 7fd20b471700 1 -- 172.21.15.33:0/3393522952 wait complete. 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 Processor -- start 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 -- start start 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204101830 0x7fd20419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 0x7fd20419ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd204106650 0x7fd204197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204198090 con 0x7fd2041021f0 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2041981d0 con 0x7fd204106650 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd20b471700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd204198390 con 0x7fd204101830 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd208a0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 0x7fd20419ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:30.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd208a0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 0x7fd20419ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34304/0 (socket says 172.21.15.33:34304) 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.311+0000 7fd208a0c700 1 -- 172.21.15.33:0/2171083714 learned_addr learned my addr 172.21.15.33:0/2171083714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd20920d700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204101830 0x7fd20419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd209a0e700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd204106650 0x7fd204197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 -- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204101830 msgr2=0x7fd20419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd204101830 0x7fd20419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 -- 172.21.15.33:0/2171083714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd204106650 msgr2=0x7fd204197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd204106650 0x7fd204197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f40095f0 con 0x7fd2041021f0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd208a0c700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 0x7fd20419ddb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f400b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f401d070 con 0x7fd2041021f0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1f400bdf0 con 0x7fd2041021f0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd204198610 con 0x7fd2041021f0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4003720 con 0x7fd2041021f0 2024-01-25T16:06:30.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.312+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd204198880 con 0x7fd2041021f0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.313+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd204102ec0 con 0x7fd2041021f0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.314+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd1f4004830 con 0x7fd2041021f0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.314+0000 7fd1fa7fc700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd1f00a1400 0x7fd1f00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd20920d700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd1f00a1400 0x7fd1f00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd1f40cda30 con 0x7fd2041021f0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd1fa7fc700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] conn(0x7fd1f00aabe0 0x7fd1f00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1f00ad6b0 con 0x7fd1f00aabe0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1f40cded0 con 0x7fd2041021f0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd20920d700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd1f00a1400 0x7fd1f00a38c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd204198b70 tx=0x7fd200009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.315+0000 7fd209a0e700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] conn(0x7fd1f00aabe0 0x7fd1f00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.316+0000 7fd209a0e700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] conn(0x7fd1f00aabe0 0x7fd1f00ad000 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.316+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== osd.11 v2:172.21.15.152:6824/249910241 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd1f00ad6b0 con 0x7fd1f00aabe0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 --> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd2041119e0 con 0x7fd1f00aabe0 2024-01-25T16:06:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd1fa7fc700 1 -- 172.21.15.33:0/2171083714 <== osd.11 v2:172.21.15.152:6824/249910241 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd2041119e0 con 0x7fd1f00aabe0 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] conn(0x7fd1f00aabe0 msgr2=0x7fd1f00ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] conn(0x7fd1f00aabe0 0x7fd1f00ad000 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd1f00a1400 msgr2=0x7fd1f00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd1f00a1400 0x7fd1f00a38c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd204198b70 tx=0x7fd200009260 comp rx=0 tx=0).stop 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 msgr2=0x7fd20419ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 --2- 172.21.15.33:0/2171083714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2041021f0 0x7fd20419ddb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4000c00 tx=0x7fd1f400b9c0 comp rx=0 tx=0).stop 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd209a0e700 1 -- 172.21.15.33:0/2171083714 reap_dead start 2024-01-25T16:06:30.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 shutdown_connections 2024-01-25T16:06:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.333+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 >> 172.21.15.33:0/2171083714 conn(0x7fd204078910 msgr2=0x7fd2040ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.334+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 shutdown_connections 2024-01-25T16:06:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:29.334+0000 7fd20b471700 1 -- 172.21.15.33:0/2171083714 wait complete. 2024-01-25T16:06:31.128 INFO:teuthology.orchestra.run.smithi033.stdout:347892350997 2024-01-25T16:06:31.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-01-25T16:06:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:31 smithi152 ceph-mon[36015]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:31 smithi076 ceph-mon[35642]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:31.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:31 smithi033 ceph-mon[31417]: pgmap v307: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:31.825 INFO:teuthology.orchestra.run.smithi033.stdout:292057776160 2024-01-25T16:06:31.825 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-01-25T16:06:32.318 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:33.257 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:33 smithi033 ceph-mon[31417]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:33 smithi076 ceph-mon[35642]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:33.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 -- 172.21.15.33:0/1189075051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0108110 msgr2=0x7f40f0101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 --2- 172.21.15.33:0/1189075051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0108110 0x7f40f0101880 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f40d8009990 tx=0x7f40d8009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 -- 172.21.15.33:0/1189075051 shutdown_connections 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 --2- 172.21.15.33:0/1189075051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f0101fb0 0x7f40f010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 --2- 172.21.15.33:0/1189075051 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f01077f0 0x7f40f0107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 --2- 172.21.15.33:0/1189075051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0108110 0x7f40f0101880 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 -- 172.21.15.33:0/1189075051 >> 172.21.15.33:0/1189075051 conn(0x7f40f00788a0 msgr2=0x7f40f0078cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 -- 172.21.15.33:0/1189075051 shutdown_connections 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.571+0000 7f40f5136700 1 -- 172.21.15.33:0/1189075051 wait complete. 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 Processor -- start 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 -- start start 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0101fb0 0x7f40f0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f01077f0 0x7f40f0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 0x7f40f019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f019d080 con 0x7f40f019c690 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f019d1f0 con 0x7f40f01077f0 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40f5136700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f019d3b0 con 0x7f40f0101fb0 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ef59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 0x7f40f019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ef59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 0x7f40f019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34322/0 (socket says 172.21.15.33:34322) 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ef59e700 1 -- 172.21.15.33:0/3775522365 learned_addr learned my addr 172.21.15.33:0/3775522365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40eed9d700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0101fb0 0x7f40f0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ee59c700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f01077f0 0x7f40f0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ef59e700 1 -- 172.21.15.33:0/3775522365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0101fb0 msgr2=0x7f40f0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.572+0000 7f40ef59e700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0101fb0 0x7f40f0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40ef59e700 1 -- 172.21.15.33:0/3775522365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f01077f0 msgr2=0x7f40f0198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40ef59e700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f01077f0 0x7f40f0198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40ef59e700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d80095f0 con 0x7f40f019c690 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40ef59e700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 0x7f40f019cb10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b970 tx=0x7f40e000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e000c930 con 0x7f40f019c690 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f01a4940 con 0x7f40f019c690 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e0014440 con 0x7f40f019c690 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e00075d0 con 0x7f40f019c690 2024-01-25T16:06:33.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.573+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f01a4df0 con 0x7f40f019c690 2024-01-25T16:06:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.574+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40f0102f20 con 0x7f40f019c690 2024-01-25T16:06:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.575+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f40e00145b0 con 0x7f40f019c690 2024-01-25T16:06:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40e7fff700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f40dc0a1400 0x7f40dc0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40eed9d700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f40dc0a1400 0x7f40dc0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f40e0006e40 con 0x7f40f019c690 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40e7fff700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] conn(0x7f40dc0aabe0 0x7f40dc0ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40dc0ad6b0 con 0x7f40dc0aabe0 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e00ccba0 con 0x7f40f019c690 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40ee59c700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] conn(0x7f40dc0aabe0 0x7f40dc0ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40eed9d700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f40dc0a1400 0x7f40dc0a38c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009a10 tx=0x7f40e8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.576+0000 7f40ee59c700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] conn(0x7f40dc0aabe0 0x7f40dc0ad000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.577+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== osd.0 v2:172.21.15.33:6802/229778098 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f40dc0ad6b0 con 0x7f40dc0aabe0 2024-01-25T16:06:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.593+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 --> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40f0111ae0 con 0x7f40dc0aabe0 2024-01-25T16:06:33.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40e7fff700 1 -- 172.21.15.33:0/3775522365 <== osd.0 v2:172.21.15.33:6802/229778098 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40f0111ae0 con 0x7f40dc0aabe0 2024-01-25T16:06:33.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] conn(0x7f40dc0aabe0 msgr2=0x7f40dc0ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] conn(0x7f40dc0aabe0 0x7f40dc0ad000 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f40dc0a1400 msgr2=0x7f40dc0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f40dc0a1400 0x7f40dc0a38c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009a10 tx=0x7f40e8009190 comp rx=0 tx=0).stop 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 msgr2=0x7f40f019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 --2- 172.21.15.33:0/3775522365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f019c690 0x7f40f019cb10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f40e000b970 tx=0x7f40e000bc80 comp rx=0 tx=0).stop 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40ef59e700 1 -- 172.21.15.33:0/3775522365 reap_dead start 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 shutdown_connections 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 >> 172.21.15.33:0/3775522365 conn(0x7f40f00788a0 msgr2=0x7f40f0100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 shutdown_connections 2024-01-25T16:06:33.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.594+0000 7f40f5136700 1 -- 172.21.15.33:0/3775522365 wait complete. 2024-01-25T16:06:33.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:33 smithi152 ceph-mon[36015]: pgmap v308: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 -- 172.21.15.33:0/1421380192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 msgr2=0x7f6c000698f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 --2- 172.21.15.33:0/1421380192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c000698f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40099c0 tx=0x7f6bf4009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 -- 172.21.15.33:0/1421380192 shutdown_connections 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 --2- 172.21.15.33:0/1421380192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c00069e30 0x7f6c0010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 --2- 172.21.15.33:0/1421380192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c001077a0 0x7f6c00107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 --2- 172.21.15.33:0/1421380192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c000698f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 -- 172.21.15.33:0/1421380192 >> 172.21.15.33:0/1421380192 conn(0x7f6c00077610 msgr2=0x7f6c00077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 -- 172.21.15.33:0/1421380192 shutdown_connections 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.746+0000 7f6c07963700 1 -- 172.21.15.33:0/1421380192 wait complete. 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 Processor -- start 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 -- start start 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c0019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c00069e30 0x7f6c0019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c001077a0 0x7f6c001979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00198050 con 0x7f6c00069490 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00198190 con 0x7f6c001077a0 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c07963700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c00198350 con 0x7f6c00069e30 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c0019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c0019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34352/0 (socket says 172.21.15.33:34352) 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 -- 172.21.15.33:0/2919125383 learned_addr learned my addr 172.21.15.33:0/2919125383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c04efe700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c00069e30 0x7f6c0019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c05f00700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c001077a0 0x7f6c001979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 -- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c00069e30 msgr2=0x7f6c0019dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6c00069e30 0x7f6c0019dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 -- 172.21.15.33:0/2919125383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c001077a0 msgr2=0x7f6c001979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c001077a0 0x7f6c001979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.747+0000 7f6c056ff700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40095f0 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6c056ff700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c0019d830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d950 tx=0x7f6bf800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf8009b40 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6bf8009ca0 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c00198630 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf800f620 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.748+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c001a48d0 con 0x7f6c00069490 2024-01-25T16:06:33.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.749+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6c00102ee0 con 0x7f6c00069490 2024-01-25T16:06:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.750+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f6bf8010460 con 0x7f6c00069490 2024-01-25T16:06:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.750+0000 7f6bfe7fc700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6be80a1400 0x7f6be80a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.750+0000 7f6c04efe700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6be80a1400 0x7f6be80a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.750+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6bf80ccff0 con 0x7f6c00069490 2024-01-25T16:06:33.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6bfe7fc700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] conn(0x7f6be80aabe0 0x7f6be80ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6be80ad6b0 con 0x7f6be80aabe0 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6bf80cd400 con 0x7f6c00069490 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6c04efe700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6be80a1400 0x7f6be80a38c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c00077cf0 tx=0x7f6bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6c05f00700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] conn(0x7f6be80aabe0 0x7f6be80ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.751+0000 7f6c05f00700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] conn(0x7f6be80aabe0 0x7f6be80ad000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:33.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.752+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== osd.8 v2:172.21.15.152:6816/281006087 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6be80ad6b0 con 0x7f6be80aabe0 2024-01-25T16:06:33.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.770+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 --> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6c00111a00 con 0x7f6be80aabe0 2024-01-25T16:06:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.770+0000 7f6bfe7fc700 1 -- 172.21.15.33:0/2919125383 <== osd.8 v2:172.21.15.152:6816/281006087 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6c00111a00 con 0x7f6be80aabe0 2024-01-25T16:06:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.770+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] conn(0x7f6be80aabe0 msgr2=0x7f6be80ad000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.770+0000 7f6c07963700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] conn(0x7f6be80aabe0 0x7f6be80ad000 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6be80a1400 msgr2=0x7f6be80a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6be80a1400 0x7f6be80a38c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c00077cf0 tx=0x7f6bf4005fd0 comp rx=0 tx=0).stop 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 msgr2=0x7f6c0019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 --2- 172.21.15.33:0/2919125383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c00069490 0x7f6c0019d830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d950 tx=0x7f6bf800dd10 comp rx=0 tx=0).stop 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c05f00700 1 -- 172.21.15.33:0/2919125383 reap_dead start 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 shutdown_connections 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 >> 172.21.15.33:0/2919125383 conn(0x7f6c00077610 msgr2=0x7f6c000fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 shutdown_connections 2024-01-25T16:06:33.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:33.771+0000 7f6c07963700 1 -- 172.21.15.33:0/2919125383 wait complete. 2024-01-25T16:06:35.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 -- 172.21.15.33:0/4047832884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef41077a0 msgr2=0x7fdef4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:35.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/4047832884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef41077a0 0x7fdef4107b80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdef00099c0 tx=0x7fdef0009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 -- 172.21.15.33:0/4047832884 shutdown_connections 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/4047832884 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/4047832884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef41077a0 0x7fdef4107b80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.200+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/4047832884 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef4069490 0x7fdef40698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- 172.21.15.33:0/4047832884 >> 172.21.15.33:0/4047832884 conn(0x7fdef4077610 msgr2=0x7fdef4077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- 172.21.15.33:0/4047832884 shutdown_connections 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- 172.21.15.33:0/4047832884 wait complete. 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 Processor -- start 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- start start 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef419de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef41077a0 0x7fdef4197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41980f0 con 0x7fdef4069490 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef4198230 con 0x7fdef41077a0 2024-01-25T16:06:35.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefc7aa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41983f0 con 0x7fdef4069e30 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.201+0000 7fdefa546700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34368/0 (socket says 172.21.15.33:34368) 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 -- 172.21.15.33:0/3186971540 learned_addr learned my addr 172.21.15.33:0/3186971540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefad47700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef41077a0 0x7fdef4197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdef9d45700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef419de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 -- 172.21.15.33:0/3186971540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 msgr2=0x7fdef419de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef419de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 -- 172.21.15.33:0/3186971540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef41077a0 msgr2=0x7fdef4197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef41077a0 0x7fdef4197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 -- 172.21.15.33:0/3186971540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00095f0 con 0x7fdef4069490 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdef9d45700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefa546700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdef00098e0 tx=0x7fdef0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:35.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000f7e0 con 0x7fdef4069490 2024-01-25T16:06:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdef0023070 con 0x7fdef4069490 2024-01-25T16:06:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef4198670 con 0x7fdef4069490 2024-01-25T16:06:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00037c0 con 0x7fdef4069490 2024-01-25T16:06:35.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.202+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef41008b0 con 0x7fdef4069490 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.203+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4109d50 con 0x7fdef4069490 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.204+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fdef0025030 con 0x7fdef4069490 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.205+0000 7fdeeb7fe700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 0x7fdee00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.205+0000 7fdef9d45700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 0x7fdee00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.205+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fdef00d1f40 con 0x7fdef4069490 2024-01-25T16:06:35.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.205+0000 7fdef9d45700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 0x7fdee00a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdef4077cf0 tx=0x7fdee4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:35.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.209+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdef009b4b0 con 0x7fdef4069490 2024-01-25T16:06:35.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.411+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdef40684a0 con 0x7fdef4069490 2024-01-25T16:06:35.414 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:35 smithi033 ceph-mon[31417]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:35.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.411+0000 7fdeeb7fe700 1 -- 172.21.15.33:0/3186971540 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdef009ead0 con 0x7fdef4069490 2024-01-25T16:06:35.414 INFO:teuthology.orchestra.run.smithi033.stdout:167503724597 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 msgr2=0x7fdee00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 0x7fdee00a3940 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdef4077cf0 tx=0x7fdee4008040 comp rx=0 tx=0).stop 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 msgr2=0x7fdef419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fdef00098e0 tx=0x7fdef0005740 comp rx=0 tx=0).stop 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 shutdown_connections 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdee00a1480 0x7fdee00a3940 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdef41077a0 0x7fdef4197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdef4069490 0x7fdef419d8d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 --2- 172.21.15.33:0/3186971540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdef4069e30 0x7fdef419de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:35.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 >> 172.21.15.33:0/3186971540 conn(0x7fdef4077610 msgr2=0x7fdef40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:35.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 shutdown_connections 2024-01-25T16:06:35.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:35.413+0000 7fdefc7aa700 1 -- 172.21.15.33:0/3186971540 wait complete. 2024-01-25T16:06:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:35 smithi076 ceph-mon[35642]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:35.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:35.446 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:35 smithi152 ceph-mon[36015]: pgmap v309: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:35.798 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:36.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3186971540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-25T16:06:36.528 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:36.537 INFO:teuthology.orchestra.run.smithi033.stdout:34359738445 2024-01-25T16:06:36.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-01-25T16:06:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3186971540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-25T16:06:36.658 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3186971540' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-01-25T16:06:37.271 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:37 smithi033 ceph-mon[31417]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:37.633 INFO:teuthology.orchestra.run.smithi033.stdout:257698037797 2024-01-25T16:06:37.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-01-25T16:06:37.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:37 smithi152 ceph-mon[36015]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:37 smithi076 ceph-mon[35642]: pgmap v310: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:39.385 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:39 smithi033 ceph-mon[31417]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-25T16:06:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:39 smithi152 ceph-mon[36015]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-25T16:06:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:39 smithi076 ceph-mon[35642]: pgmap v311: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-01-25T16:06:40.161 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724592 got 167503724597 for osd.5 2024-01-25T16:06:40.161 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.017+0000 7fb6df728700 1 -- 172.21.15.33:0/3195141491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8108110 msgr2=0x7fb6d8101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.017+0000 7fb6df728700 1 --2- 172.21.15.33:0/3195141491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8108110 0x7fb6d8101880 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009990 tx=0x7fb6cc009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 -- 172.21.15.33:0/3195141491 shutdown_connections 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 --2- 172.21.15.33:0/3195141491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 --2- 172.21.15.33:0/3195141491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 0x7fb6d8107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 --2- 172.21.15.33:0/3195141491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8108110 0x7fb6d8101880 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 -- 172.21.15.33:0/3195141491 >> 172.21.15.33:0/3195141491 conn(0x7fb6d80788a0 msgr2=0x7fb6d8078cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 -- 172.21.15.33:0/3195141491 shutdown_connections 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 -- 172.21.15.33:0/3195141491 wait complete. 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 Processor -- start 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.018+0000 7fb6df728700 1 -- start start 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 0x7fb6d819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:41.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8198500 con 0x7fb6d8197ab0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8198670 con 0x7fb6d8101fb0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6df728700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8198830 con 0x7fb6d81077f0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58014/0 (socket says 172.21.15.33:58014) 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 -- 172.21.15.33:0/3375197223 learned_addr learned my addr 172.21.15.33:0/3375197223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6dccc3700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 0x7fb6d819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6dd4c4700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 -- 172.21.15.33:0/3375197223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 msgr2=0x7fb6d819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 0x7fb6d819de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 -- 172.21.15.33:0/3375197223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 msgr2=0x7fb6d819d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d819d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 -- 172.21.15.33:0/3375197223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0095f0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6dd4c4700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6ddcc5700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000e940 tx=0x7fb6d000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00097b0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.019+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6d000ee90 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.020+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d81a4820 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.020+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0005520 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.020+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81a4cf0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.021+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d8109d50 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.023+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fb6d0004750 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.023+0000 7fb6d67fc700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 0x7fb6c00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.024+0000 7fb6dd4c4700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 0x7fb6c00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.024+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb6d0014070 con 0x7fb6d8197ab0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.024+0000 7fb6dd4c4700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 0x7fb6c00a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8198bf0 tx=0x7fb6c8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.028+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb6d00963b0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.239+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb6d8198fe0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.240+0000 7fb6d67fc700 1 -- 172.21.15.33:0/3375197223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6d00999d0 con 0x7fb6d8197ab0 2024-01-25T16:06:41.242 INFO:teuthology.orchestra.run.smithi033.stdout:223338299435 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 msgr2=0x7fb6c00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 0x7fb6c00a3940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8198bf0 tx=0x7fb6c8009040 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 msgr2=0x7fb6d8197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000e940 tx=0x7fb6d000ec50 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 shutdown_connections 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fb6c00a1480 0x7fb6c00a3940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6d8197ab0 0x7fb6d8197f30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6d8101fb0 0x7fb6d819d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 --2- 172.21.15.33:0/3375197223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb6d81077f0 0x7fb6d819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 >> 172.21.15.33:0/3375197223 conn(0x7fb6d80788a0 msgr2=0x7fb6d8100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 shutdown_connections 2024-01-25T16:06:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:41.242+0000 7fb6df728700 1 -- 172.21.15.33:0/3375197223 wait complete. 2024-01-25T16:06:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:41 smithi152 ceph-mon[36015]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:41.412 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:41 smithi033 ceph-mon[31417]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:41 smithi076 ceph-mon[35642]: pgmap v312: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:42.155 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3375197223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-25T16:06:42.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3375197223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-25T16:06:42.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3375197223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-01-25T16:06:43.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:43 smithi033 ceph-mon[31417]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:43 smithi152 ceph-mon[36015]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:43 smithi076 ceph-mon[35642]: pgmap v313: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:06:44.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 -- 172.21.15.33:0/3167878486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc100790 msgr2=0x7f89cc10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 --2- 172.21.15.33:0/3167878486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc100790 0x7f89cc10e6f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009990 tx=0x7f89c4009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 -- 172.21.15.33:0/3167878486 shutdown_connections 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 --2- 172.21.15.33:0/3167878486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc100790 0x7f89cc10e6f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 --2- 172.21.15.33:0/3167878486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc106850 0x7f89cc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 --2- 172.21.15.33:0/3167878486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 0x7f89cc100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 -- 172.21.15.33:0/3167878486 >> 172.21.15.33:0/3167878486 conn(0x7f89cc076ce0 msgr2=0x7f89cc0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 -- 172.21.15.33:0/3167878486 shutdown_connections 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.682+0000 7f89d3e15700 1 -- 172.21.15.33:0/3167878486 wait complete. 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 Processor -- start 2024-01-25T16:06:44.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 -- start start 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 0x7f89cc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc100790 0x7f89cc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc19ccb0 con 0x7f89cc106850 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc19ce20 con 0x7f89cc100790 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d3e15700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89cc19cfe0 con 0x7f89cc0ffdd0 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58024/0 (socket says 172.21.15.33:58024) 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 -- 172.21.15.33:0/1240243569 learned_addr learned my addr 172.21.15.33:0/1240243569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d1bb1700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 0x7f89cc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d13b0700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc100790 0x7f89cc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 -- 172.21.15.33:0/1240243569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 msgr2=0x7f89cc197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:44.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 0x7f89cc197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 -- 172.21.15.33:0/1240243569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc100790 msgr2=0x7f89cc198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc100790 0x7f89cc198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 -- 172.21.15.33:0/1240243569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40095f0 con 0x7f89cc106850 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.683+0000 7f89d23b2700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005910 tx=0x7f89c400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.684+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c401d070 con 0x7f89cc106850 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.684+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89c400fb70 con 0x7f89cc106850 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.684+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89cc19d260 con 0x7f89cc106850 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.684+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c4017650 con 0x7f89cc106850 2024-01-25T16:06:44.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.684+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89cc1a4b20 con 0x7f89cc106850 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.685+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89cc109e40 con 0x7f89cc106850 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.686+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f89c400fce0 con 0x7f89cc106850 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.686+0000 7f89caffd700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 0x7f89b40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.686+0000 7f89d1bb1700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 0x7f89b40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.686+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f89c40ce190 con 0x7f89cc106850 2024-01-25T16:06:44.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.687+0000 7f89d1bb1700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 0x7f89b40a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0060b0 tx=0x7f89bc00c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:44.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.690+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f89c40976f0 con 0x7f89cc106850 2024-01-25T16:06:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.889+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f89cc068500 con 0x7f89cc106850 2024-01-25T16:06:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.889+0000 7f89caffd700 1 -- 172.21.15.33:0/1240243569 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f89c4027090 con 0x7f89cc106850 2024-01-25T16:06:44.892 INFO:teuthology.orchestra.run.smithi033.stdout:313532612637 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 msgr2=0x7f89b40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 0x7f89b40a3940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f89bc0060b0 tx=0x7f89bc00c040 comp rx=0 tx=0).stop 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 msgr2=0x7f89cc19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f89c4005910 tx=0x7f89c400f690 comp rx=0 tx=0).stop 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 shutdown_connections 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89b40a1480 0x7f89b40a3940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89cc106850 0x7f89cc19c6a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.891+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89cc0ffdd0 0x7f89cc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.892+0000 7f89d3e15700 1 --2- 172.21.15.33:0/1240243569 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89cc100790 0x7f89cc198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.892+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 >> 172.21.15.33:0/1240243569 conn(0x7f89cc076ce0 msgr2=0x7f89cc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.892+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 shutdown_connections 2024-01-25T16:06:44.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:44.892+0000 7f89d3e15700 1 -- 172.21.15.33:0/1240243569 wait complete. 2024-01-25T16:06:45.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:45 smithi033 ceph-mon[31417]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:45.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:45 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1240243569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-25T16:06:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 -- 172.21.15.33:0/2816157716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 msgr2=0x7f7d6810e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:45.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/2816157716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d6810e6f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000b0a0 tx=0x7f7d6000b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 -- 172.21.15.33:0/2816157716 shutdown_connections 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/2816157716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d6810e6f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/2816157716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d68106850 0x7f7d68106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/2816157716 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d680ffdd0 0x7f7d68100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.542+0000 7f7d6ffac700 1 -- 172.21.15.33:0/2816157716 >> 172.21.15.33:0/2816157716 conn(0x7f7d68076ce0 msgr2=0x7f7d680fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.543+0000 7f7d6ffac700 1 -- 172.21.15.33:0/2816157716 shutdown_connections 2024-01-25T16:06:45.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.543+0000 7f7d6ffac700 1 -- 172.21.15.33:0/2816157716 wait complete. 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.543+0000 7f7d6ffac700 1 Processor -- start 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.543+0000 7f7d6ffac700 1 -- start start 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.543+0000 7f7d6ffac700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d680ffdd0 0x7f7d68100fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68106850 0x7f7d68101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681a6190 con 0x7f7d68100790 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681a6300 con 0x7f7d680ffdd0 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d681a64c0 con 0x7f7d68106850 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:45.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58040/0 (socket says 172.21.15.33:58040) 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 -- 172.21.15.33:0/1753585641 learned_addr learned my addr 172.21.15.33:0/1753585641 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6e549700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68106850 0x7f7d68101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6dd48700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d680ffdd0 0x7f7d68100fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 -- 172.21.15.33:0/1753585641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68106850 msgr2=0x7f7d68101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68106850 0x7f7d68101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 -- 172.21.15.33:0/1753585641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d680ffdd0 msgr2=0x7f7d68100fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d680ffdd0 0x7f7d68100fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 -- 172.21.15.33:0/1753585641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d60009d00 con 0x7f7d68100790 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6d547700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c9b0 tx=0x7f7d5800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58007b00 con 0x7f7d68100790 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d681a67a0 con 0x7f7d68100790 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d58007c60 con 0x7f7d68100790 2024-01-25T16:06:45.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d58018660 con 0x7f7d68100790 2024-01-25T16:06:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.544+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d681a6ca0 con 0x7f7d68100790 2024-01-25T16:06:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.545+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d68109d70 con 0x7f7d68100790 2024-01-25T16:06:45.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.546+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f7d5800f450 con 0x7f7d68100790 2024-01-25T16:06:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.547+0000 7f7d5effd700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 0x7f7d540a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.547+0000 7f7d6dd48700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 0x7f7d540a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.547+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7d580ccef0 con 0x7f7d68100790 2024-01-25T16:06:45.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.547+0000 7f7d6dd48700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 0x7f7d540a38c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005950 tx=0x7f7d640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:45.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.551+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7d58096350 con 0x7f7d68100790 2024-01-25T16:06:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:45 smithi152 ceph-mon[36015]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:45 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1240243569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-25T16:06:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:45 smithi076 ceph-mon[35642]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:06:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:45 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1240243569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-01-25T16:06:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.760+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7d681030d0 con 0x7f7d68100790 2024-01-25T16:06:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.760+0000 7f7d5effd700 1 -- 172.21.15.33:0/1753585641 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d58099970 con 0x7f7d68100790 2024-01-25T16:06:45.763 INFO:teuthology.orchestra.run.smithi033.stdout:141733920827 2024-01-25T16:06:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 msgr2=0x7f7d540a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 0x7f7d540a38c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7d64005950 tx=0x7f7d640058e0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 msgr2=0x7f7d681014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800c9b0 tx=0x7f7d5800ccc0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 shutdown_connections 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d540a1400 0x7f7d540a38c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d68106850 0x7f7d68101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d680ffdd0 0x7f7d68100fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 --2- 172.21.15.33:0/1753585641 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d68100790 0x7f7d681014f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 >> 172.21.15.33:0/1753585641 conn(0x7f7d68076ce0 msgr2=0x7f7d680fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 shutdown_connections 2024-01-25T16:06:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:45.763+0000 7f7d6ffac700 1 -- 172.21.15.33:0/1753585641 wait complete. 2024-01-25T16:06:46.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:46 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:06:46.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:46 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1753585641' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-25T16:06:46.424 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:46 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:06:46.424 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:46 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1753585641' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-25T16:06:46.442 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:46 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:06:46.442 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:46 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1753585641' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/4199030591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98106650 msgr2=0x7f4a98106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/4199030591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98106650 0x7f4a98106a30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a940099c0 tx=0x7f4a94009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/4199030591 shutdown_connections 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/4199030591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a981021f0 0x7f4a9810e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/4199030591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98106650 0x7f4a98106a30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/4199030591 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a98101830 0x7f4a98101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.730+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/4199030591 >> 172.21.15.33:0/4199030591 conn(0x7f4a98078910 msgr2=0x7f4a98078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/4199030591 shutdown_connections 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/4199030591 wait complete. 2024-01-25T16:06:46.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 Processor -- start 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- start start 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a981021f0 0x7f4a98074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a98106650 0x7f4a98074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a981a6040 con 0x7f4a98101830 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a981a61b0 con 0x7f4a98106650 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9fb2a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a981a6370 con 0x7f4a981021f0 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9d8c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9d8c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58066/0 (socket says 172.21.15.33:58066) 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.731+0000 7f4a9d8c6700 1 -- 172.21.15.33:0/1104065023 learned_addr learned my addr 172.21.15.33:0/1104065023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9e0c7700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a98106650 0x7f4a98074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d0c5700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a981021f0 0x7f4a98074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:46.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 -- 172.21.15.33:0/1104065023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a981021f0 msgr2=0x7f4a98074020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a981021f0 0x7f4a98074020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 -- 172.21.15.33:0/1104065023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a98106650 msgr2=0x7f4a98074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a98106650 0x7f4a98074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 -- 172.21.15.33:0/1104065023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a940095f0 con 0x7f4a98101830 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9d8c6700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b580 tx=0x7f4a94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9401d070 con 0x7f4a98101830 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a9400fb30 con 0x7f4a98101830 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a981a65f0 con 0x7f4a98101830 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94022c80 con 0x7f4a98101830 2024-01-25T16:06:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.732+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a981a6a00 con 0x7f4a98101830 2024-01-25T16:06:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.733+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a98109d10 con 0x7f4a98101830 2024-01-25T16:06:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.734+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f4a9400fca0 con 0x7f4a98101830 2024-01-25T16:06:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.735+0000 7f4a8affd700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 0x7f4a840a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.735+0000 7f4a9d0c5700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 0x7f4a840a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.735+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4a940cd930 con 0x7f4a98101830 2024-01-25T16:06:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.735+0000 7f4a9d0c5700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 0x7f4a840a38c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a98078f80 tx=0x7f4a8c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.739+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4a94096e90 con 0x7f4a98101830 2024-01-25T16:06:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.942+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f4a980684a0 con 0x7f4a98101830 2024-01-25T16:06:46.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.942+0000 7f4a8affd700 1 -- 172.21.15.33:0/1104065023 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4a9409a4b0 con 0x7f4a98101830 2024-01-25T16:06:46.945 INFO:teuthology.orchestra.run.smithi033.stdout:347892351000 2024-01-25T16:06:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 msgr2=0x7f4a840a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 0x7f4a840a38c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4a98078f80 tx=0x7f4a8c00b3c0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 msgr2=0x7f4a98073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400b580 tx=0x7f4a94004730 comp rx=0 tx=0).stop 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 shutdown_connections 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4a840a1400 0x7f4a840a38c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a98106650 0x7f4a98074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a98101830 0x7f4a98073ae0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 --2- 172.21.15.33:0/1104065023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4a981021f0 0x7f4a98074020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.944+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 >> 172.21.15.33:0/1104065023 conn(0x7f4a98078910 msgr2=0x7f4a980ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.945+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 shutdown_connections 2024-01-25T16:06:46.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:46.945+0000 7f4a9fb2a700 1 -- 172.21.15.33:0/1104065023 wait complete. 2024-01-25T16:06:47.191 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612633 got 313532612637 for osd.10 2024-01-25T16:06:47.192 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:47.445 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:47 smithi033 ceph-mon[31417]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:47.446 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1104065023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-25T16:06:47.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299430 got 223338299435 for osd.7 2024-01-25T16:06:47.584 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:47 smithi152 ceph-mon[36015]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1104065023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-25T16:06:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:47 smithi076 ceph-mon[35642]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:06:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1104065023' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-01-25T16:06:48.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.548 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-25T16:06:48.549 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:48 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-01-25T16:06:48.574 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920822 got 141733920827 for osd.4 2024-01-25T16:06:48.574 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-25T16:06:48.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:48 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-01-25T16:06:48.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-01-25T16:06:48.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:48 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]: dispatch 2024-01-25T16:06:49.309 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350997 got 347892351000 for osd.11 2024-01-25T16:06:49.309 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-01-25T16:06:49.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:49 smithi033 ceph-mon[31417]: osdmap e87: 12 total, 12 up, 12 in 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-25T16:06:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-25T16:06:49.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-01-25T16:06:49.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:49 smithi152 ceph-mon[36015]: osdmap e87: 12 total, 12 up, 12 in 2024-01-25T16:06:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: pgmap v316: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 8]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 7]}]': finished 2024-01-25T16:06:49.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:49 smithi076 ceph-mon[35642]: osdmap e87: 12 total, 12 up, 12 in 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 -- 172.21.15.33:0/2474023803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c100750 msgr2=0x7f0f9c100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/2474023803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c100750 0x7f0f9c100bd0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009990 tx=0x7f0f8c009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 -- 172.21.15.33:0/2474023803 shutdown_connections 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/2474023803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c101110 0x7f0f9c10d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/2474023803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c106750 0x7f0f9c106b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/2474023803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c100750 0x7f0f9c100bd0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 -- 172.21.15.33:0/2474023803 >> 172.21.15.33:0/2474023803 conn(0x7f0f9c077810 msgr2=0x7f0f9c077c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 -- 172.21.15.33:0/2474023803 shutdown_connections 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.791+0000 7f0fa3edc700 1 -- 172.21.15.33:0/2474023803 wait complete. 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 Processor -- start 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 -- start start 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c100750 0x7f0f9c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c101110 0x7f0f9c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c1999c0 con 0x7f0f9c106750 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c1a4180 con 0x7f0f9c100750 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa3edc700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9c1a42c0 con 0x7f0f9c101110 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa2479700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:49.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa2479700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32836/0 (socket says 172.21.15.33:32836) 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.792+0000 7f0fa2479700 1 -- 172.21.15.33:0/186590801 learned_addr learned my addr 172.21.15.33:0/186590801 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa1c78700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c100750 0x7f0f9c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa1477700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c101110 0x7f0f9c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 -- 172.21.15.33:0/186590801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c101110 msgr2=0x7f0f9c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c101110 0x7f0f9c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 -- 172.21.15.33:0/186590801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c100750 msgr2=0x7f0f9c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c100750 0x7f0f9c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 -- 172.21.15.33:0/186590801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c0095f0 con 0x7f0f9c106750 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa2479700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f94007ee0 tx=0x7f0f9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94017070 con 0x7f0f9c106750 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9400f040 con 0x7f0f9c106750 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9c1a45a0 con 0x7f0f9c106750 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f94005740 con 0x7f0f9c106750 2024-01-25T16:06:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.793+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9c1a4a50 con 0x7f0f9c106750 2024-01-25T16:06:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.794+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9c108ce0 con 0x7f0f9c106750 2024-01-25T16:06:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.795+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f0f9400dc60 con 0x7f0f9c106750 2024-01-25T16:06:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.796+0000 7f0f92ffd700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 0x7f0f880a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.796+0000 7f0fa1c78700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 0x7f0f880a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.796+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0f940cc520 con 0x7f0f9c106750 2024-01-25T16:06:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.796+0000 7f0fa1c78700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 0x7f0f880a38c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005950 tx=0x7f0f9800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:49.800+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0f94095840 con 0x7f0f9c106750 2024-01-25T16:06:50.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.010+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0f9c06a530 con 0x7f0f9c106750 2024-01-25T16:06:50.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.010+0000 7f0f92ffd700 1 -- 172.21.15.33:0/186590801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0f94098e60 con 0x7f0f9c106750 2024-01-25T16:06:50.014 INFO:teuthology.orchestra.run.smithi033.stdout:77309411398 2024-01-25T16:06:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.012+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 msgr2=0x7f0f880a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.012+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 0x7f0f880a38c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f98005950 tx=0x7f0f9800b410 comp rx=0 tx=0).stop 2024-01-25T16:06:50.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.012+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 msgr2=0x7f0f9c1a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.012+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f94007ee0 tx=0x7f0f9400d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 shutdown_connections 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0f880a1400 0x7f0f880a38c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f9c106750 0x7f0f9c1a3c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f9c100750 0x7f0f9c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:50.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 --2- 172.21.15.33:0/186590801 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0f9c101110 0x7f0f9c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 >> 172.21.15.33:0/186590801 conn(0x7f0f9c077810 msgr2=0x7f0f9c103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 shutdown_connections 2024-01-25T16:06:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:50.013+0000 7f0fa3edc700 1 -- 172.21.15.33:0/186590801 wait complete. 2024-01-25T16:06:50.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:50 smithi033 ceph-mon[31417]: osdmap e88: 12 total, 12 up, 12 in 2024-01-25T16:06:50.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:50 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/186590801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-25T16:06:50.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:50 smithi033 ceph-mon[31417]: pgmap v319: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-01-25T16:06:50.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:50 smithi152 ceph-mon[36015]: osdmap e88: 12 total, 12 up, 12 in 2024-01-25T16:06:50.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:50 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/186590801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-25T16:06:50.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:50 smithi152 ceph-mon[36015]: pgmap v319: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-01-25T16:06:50.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:50 smithi076 ceph-mon[35642]: osdmap e88: 12 total, 12 up, 12 in 2024-01-25T16:06:50.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:50 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/186590801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-01-25T16:06:50.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:50 smithi076 ceph-mon[35642]: pgmap v319: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-01-25T16:06:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 -- 172.21.15.33:0/916464623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc101830 msgr2=0x7f03fc101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 --2- 172.21.15.33:0/916464623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc101830 0x7f03fc101cb0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f03f00099c0 tx=0x7f03f0009cd0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 -- 172.21.15.33:0/916464623 shutdown_connections 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 --2- 172.21.15.33:0/916464623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc1021f0 0x7f03fc10e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 --2- 172.21.15.33:0/916464623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc106650 0x7f03fc106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 --2- 172.21.15.33:0/916464623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc101830 0x7f03fc101cb0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 -- 172.21.15.33:0/916464623 >> 172.21.15.33:0/916464623 conn(0x7f03fc078910 msgr2=0x7f03fc078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 -- 172.21.15.33:0/916464623 shutdown_connections 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.144+0000 7f04011be700 1 -- 172.21.15.33:0/916464623 wait complete. 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.145+0000 7f04011be700 1 Processor -- start 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.145+0000 7f04011be700 1 -- start start 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.145+0000 7f04011be700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 0x7f03fc19d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc106650 0x7f03fc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc198090 con 0x7f03fc1021f0 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1981d0 con 0x7f03fc106650 2024-01-25T16:06:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc198390 con 0x7f03fc101830 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 0x7f03fc19d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fad9d700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 0x7f03fc19d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:59134/0 (socket says 172.21.15.33:59134) 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32854/0 (socket says 172.21.15.33:32854) 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fad9d700 1 -- 172.21.15.33:0/1823058838 learned_addr learned my addr 172.21.15.33:0/1823058838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fb59e700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc106650 0x7f03fc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 -- 172.21.15.33:0/1823058838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 msgr2=0x7f03fc19d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 0x7f03fc19d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 -- 172.21.15.33:0/1823058838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc106650 msgr2=0x7f03fc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc106650 0x7f03fc197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 -- 172.21.15.33:0/1823058838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00095f0 con 0x7f03fc1021f0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03fa59c700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009f50 tx=0x7f03f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f001d070 con 0x7f03fc1021f0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc198610 con 0x7f03fc1021f0 2024-01-25T16:06:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03f0004ca0 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f000f690 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.146+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1988a0 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.147+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc109d10 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.148+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f03f0005020 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.149+0000 7f03e3fff700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 0x7f03e40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.149+0000 7f03fad9d700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 0x7f03e40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.149+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f03f00cd490 con 0x7f03fc1021f0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.150+0000 7f03fad9d700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 0x7f03e40a38c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03fc198b70 tx=0x7f03ec009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.154+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f03f0096840 con 0x7f03fc1021f0 2024-01-25T16:06:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.356+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03fc1988a0 con 0x7f03fc1021f0 2024-01-25T16:06:51.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.357+0000 7f03e3fff700 1 -- 172.21.15.33:0/1823058838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f03fc1988a0 con 0x7f03fc1021f0 2024-01-25T16:06:51.360 INFO:teuthology.orchestra.run.smithi033.stdout:55834574923 2024-01-25T16:06:51.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 msgr2=0x7f03e40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:51.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 0x7f03e40a38c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f03fc198b70 tx=0x7f03ec009040 comp rx=0 tx=0).stop 2024-01-25T16:06:51.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 msgr2=0x7f03fc19ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009f50 tx=0x7f03f0004730 comp rx=0 tx=0).stop 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 shutdown_connections 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03e40a1400 0x7f03e40a38c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc106650 0x7f03fc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03fc101830 0x7f03fc19d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 --2- 172.21.15.33:0/1823058838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03fc1021f0 0x7f03fc19ddb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 >> 172.21.15.33:0/1823058838 conn(0x7f03fc078910 msgr2=0x7f03fc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 shutdown_connections 2024-01-25T16:06:51.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.359+0000 7f04011be700 1 -- 172.21.15.33:0/1823058838 wait complete. 2024-01-25T16:06:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:52 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1823058838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-25T16:06:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:52 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2131540673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-25T16:06:52.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:52 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1823058838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-25T16:06:52.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:52 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2131540673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-25T16:06:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:52 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1823058838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-01-25T16:06:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:52 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2131540673' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-01-25T16:06:52.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411398 for osd.2 2024-01-25T16:06:52.509 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 -- 172.21.15.33:0/1456419499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf441005c0 msgr2=0x7fbf44100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/1456419499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf441005c0 0x7fbf44100a40 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009990 tx=0x7fbf34009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 -- 172.21.15.33:0/1456419499 shutdown_connections 2024-01-25T16:06:52.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/1456419499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf44100f80 0x7fbf4410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/1456419499 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44106650 0x7fbf44106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/1456419499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf441005c0 0x7fbf44100a40 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 -- 172.21.15.33:0/1456419499 >> 172.21.15.33:0/1456419499 conn(0x7fbf440fc0b0 msgr2=0x7fbf440fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 -- 172.21.15.33:0/1456419499 shutdown_connections 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.788+0000 7fbf4a79e700 1 -- 172.21.15.33:0/1456419499 wait complete. 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 Processor -- start 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 -- start start 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf441005c0 0x7fbf441966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44100f80 0x7fbf44196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441a5de0 con 0x7fbf44106650 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441a5f50 con 0x7fbf441005c0 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4a79e700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441a6110 con 0x7fbf44100f80 2024-01-25T16:06:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf49f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf49f9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32878/0 (socket says 172.21.15.33:32878) 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf49f9d700 1 -- 172.21.15.33:0/2131540673 learned_addr learned my addr 172.21.15.33:0/2131540673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf48f9b700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44100f80 0x7fbf44196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stdout:107374182466 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.789+0000 7fbf4979c700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf441005c0 0x7fbf441966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:52.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 -- 172.21.15.33:0/2131540673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44100f80 msgr2=0x7fbf44196be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44100f80 0x7fbf44196be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 -- 172.21.15.33:0/2131540673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf441005c0 msgr2=0x7fbf441966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf441005c0 0x7fbf441966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 -- 172.21.15.33:0/2131540673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf49f9d700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c007ee0 tx=0x7fbf3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf3c00f040 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c005580 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf441a63f0 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.790+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf441a68a0 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.791+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf44108be0 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.792+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fbf3c00db20 con 0x7fbf44106650 2024-01-25T16:06:52.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.793+0000 7fbf3a7fc700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 0x7fbf300a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.793+0000 7fbf4979c700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 0x7fbf300a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.793+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbf3c0cca30 con 0x7fbf44106650 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.793+0000 7fbf4979c700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 0x7fbf300a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf44101700 tx=0x7fbf4000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:51.796+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbf3c095d10 con 0x7fbf44106650 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.000+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbf4406a310 con 0x7fbf44106650 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.000+0000 7fbf3a7fc700 1 -- 172.21.15.33:0/2131540673 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf3c099330 con 0x7fbf44106650 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 msgr2=0x7fbf300a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 0x7fbf300a3940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbf44101700 tx=0x7fbf4000b410 comp rx=0 tx=0).stop 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 msgr2=0x7fbf44197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c007ee0 tx=0x7fbf3c00d3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 shutdown_connections 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbf300a1480 0x7fbf300a3940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf44106650 0x7fbf44197120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf441005c0 0x7fbf441966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 --2- 172.21.15.33:0/2131540673 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbf44100f80 0x7fbf44196be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 >> 172.21.15.33:0/2131540673 conn(0x7fbf440fc0b0 msgr2=0x7fbf440fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 shutdown_connections 2024-01-25T16:06:52.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:52.002+0000 7fbf4a79e700 1 -- 172.21.15.33:0/2131540673 wait complete. 2024-01-25T16:06:53.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:53 smithi033 ceph-mon[31417]: pgmap v320: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:53.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182466 for osd.3 2024-01-25T16:06:53.116 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 -- 172.21.15.33:0/81619806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc1010d0 msgr2=0x7f5bdc10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/81619806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc10d530 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b0a0 tx=0x7f5bd400b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 -- 172.21.15.33:0/81619806 shutdown_connections 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/81619806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc10d530 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/81619806 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1066d0 0x7f5bdc106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/81619806 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc100710 0x7f5bdc100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 -- 172.21.15.33:0/81619806 >> 172.21.15.33:0/81619806 conn(0x7f5bdc076700 msgr2=0x7f5bdc076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 -- 172.21.15.33:0/81619806 shutdown_connections 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.338+0000 7f5be3bf3700 1 -- 172.21.15.33:0/81619806 wait complete. 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 Processor -- start 2024-01-25T16:06:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 -- start start 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc1066d0 0x7f5bdc1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc199940 con 0x7f5bdc100710 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1a40c0 con 0x7f5bdc1066d0 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be3bf3700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1a4230 con 0x7f5bdc1010d0 2024-01-25T16:06:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be198f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be198f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32896/0 (socket says 172.21.15.33:32896) 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.339+0000 7f5be198f700 1 -- 172.21.15.33:0/2644951671 learned_addr learned my addr 172.21.15.33:0/2644951671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be118e700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 -- 172.21.15.33:0/2644951671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1010d0 msgr2=0x7f5bdc199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 -- 172.21.15.33:0/2644951671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc1066d0 msgr2=0x7f5bdc1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc1066d0 0x7f5bdc1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 -- 172.21.15.33:0/2644951671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4009d00 con 0x7f5bdc100710 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be198f700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d950 tx=0x7f5bd800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8009ae0 con 0x7f5bdc100710 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd8009c40 con 0x7f5bdc100710 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc1a4510 con 0x7f5bdc100710 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd800f620 con 0x7f5bdc100710 2024-01-25T16:06:53.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.340+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc1a4a60 con 0x7f5bdc100710 2024-01-25T16:06:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.341+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc108bf0 con 0x7f5bdc100710 2024-01-25T16:06:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.342+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5bd8010460 con 0x7f5bdc100710 2024-01-25T16:06:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.343+0000 7f5bd2ffd700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.343+0000 7f5be118e700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.343+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5bd80ccaf0 con 0x7f5bdc100710 2024-01-25T16:06:53.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.343+0000 7f5be118e700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 0x7f5bc80a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005950 tx=0x7f5bcc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:53.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.348+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5bd8095dd0 con 0x7f5bdc100710 2024-01-25T16:06:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:06:53.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:53 smithi152 ceph-mon[36015]: pgmap v320: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:53 smithi076 ceph-mon[35642]: pgmap v320: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:53.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.557+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5bdc1106a0 con 0x7f5bdc100710 2024-01-25T16:06:53.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.557+0000 7f5bd2ffd700 1 -- 172.21.15.33:0/2644951671 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5bd80993f0 con 0x7f5bdc100710 2024-01-25T16:06:53.560 INFO:teuthology.orchestra.run.smithi033.stdout:201863462964 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 msgr2=0x7f5bc80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 0x7f5bc80a3940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005950 tx=0x7f5bcc0058e0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 msgr2=0x7f5bdc198b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800d950 tx=0x7f5bd800dd10 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 shutdown_connections 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5bc80a1480 0x7f5bc80a3940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc1066d0 0x7f5bdc1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bdc100710 0x7f5bdc198b00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 --2- 172.21.15.33:0/2644951671 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bdc1010d0 0x7f5bdc199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 >> 172.21.15.33:0/2644951671 conn(0x7f5bdc076700 msgr2=0x7f5bdc0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 shutdown_connections 2024-01-25T16:06:53.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:53.559+0000 7f5be3bf3700 1 -- 172.21.15.33:0/2644951671 wait complete. 2024-01-25T16:06:53.772 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574923 for osd.1 2024-01-25T16:06:53.772 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:54.352 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:54 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2644951671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-25T16:06:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:54 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2644951671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-25T16:06:54.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:54 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2644951671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-01-25T16:06:54.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 -- 172.21.15.33:0/2420926342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c1077a0 msgr2=0x7f971c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:54.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 --2- 172.21.15.33:0/2420926342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c1077a0 0x7f971c107b80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b0d0 tx=0x7f970c00b3e0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 -- 172.21.15.33:0/2420926342 shutdown_connections 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 --2- 172.21.15.33:0/2420926342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 0x7f971c10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 --2- 172.21.15.33:0/2420926342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c1077a0 0x7f971c107b80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 --2- 172.21.15.33:0/2420926342 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c069490 0x7f971c0698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 -- 172.21.15.33:0/2420926342 >> 172.21.15.33:0/2420926342 conn(0x7f971c077610 msgr2=0x7f971c077a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 -- 172.21.15.33:0/2420926342 shutdown_connections 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.671+0000 7f9720c21700 1 -- 172.21.15.33:0/2420926342 wait complete. 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 Processor -- start 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 -- start start 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 0x7f971c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:54.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c1077a0 0x7f971c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1a60a0 con 0x7f971c069490 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1a6210 con 0x7f971c069e30 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9720c21700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1a63d0 con 0x7f971c1077a0 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32916/0 (socket says 172.21.15.33:32916) 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 -- 172.21.15.33:0/2413173053 learned_addr learned my addr 172.21.15.33:0/2413173053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971ad9d700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c1077a0 0x7f971c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f9719d9b700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 0x7f971c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 -- 172.21.15.33:0/2413173053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c1077a0 msgr2=0x7f971c101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c1077a0 0x7f971c101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 -- 172.21.15.33:0/2413173053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 msgr2=0x7f971c100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 0x7f971c100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 -- 172.21.15.33:0/2413173053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c009d00 con 0x7f971c069490 2024-01-25T16:06:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.672+0000 7f971a59c700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f970c0094f0 tx=0x7f970c009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.673+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c00e040 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.673+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f970c005070 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.673+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c1a6650 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.673+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c01b6d0 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.673+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c1a6ac0 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.674+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c109d50 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.675+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f970c003a00 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.675+0000 7f97137fe700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 0x7f97080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.676+0000 7f9719d9b700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 0x7f97080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.676+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f970c0ce130 con 0x7f971c069490 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.676+0000 7f9719d9b700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 0x7f97080a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f971c077a30 tx=0x7f970400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:54.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.679+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f970c0975c0 con 0x7f971c069490 2024-01-25T16:06:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.883+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f971c101ce0 con 0x7f971c069490 2024-01-25T16:06:54.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.883+0000 7f97137fe700 1 -- 172.21.15.33:0/2413173053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f970c01f020 con 0x7f971c069490 2024-01-25T16:06:54.886 INFO:teuthology.orchestra.run.smithi033.stdout:292057776165 2024-01-25T16:06:54.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 msgr2=0x7f97080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 0x7f97080a3940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f971c077a30 tx=0x7f970400b410 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 msgr2=0x7f971c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f970c0094f0 tx=0x7f970c009520 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 shutdown_connections 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f97080a1480 0x7f97080a3940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c1077a0 0x7f971c101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f971c069490 0x7f971c1007b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 --2- 172.21.15.33:0/2413173053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f971c069e30 0x7f971c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 >> 172.21.15.33:0/2413173053 conn(0x7f971c077610 msgr2=0x7f971c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 shutdown_connections 2024-01-25T16:06:54.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:54.885+0000 7f9720c21700 1 -- 172.21.15.33:0/2413173053 wait complete. 2024-01-25T16:06:55.099 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462958 got 201863462964 for osd.6 2024-01-25T16:06:55.099 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:55.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:55 smithi033 ceph-mon[31417]: pgmap v321: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:55.344 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:55 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2413173053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-25T16:06:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:55 smithi152 ceph-mon[36015]: pgmap v321: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:55 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2413173053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-25T16:06:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:55 smithi076 ceph-mon[35642]: pgmap v321: 97 pgs: 1 activating, 96 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:55 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2413173053' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-01-25T16:06:55.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776165 for osd.9 2024-01-25T16:06:55.679 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:57.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:57 smithi033 ceph-mon[31417]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:57 smithi152 ceph-mon[36015]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:57 smithi076 ceph-mon[35642]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 4 B/s, 2 keys/s, 0 objects/s recovering 2024-01-25T16:06:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 -- 172.21.15.33:0/1517911858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1018a0 msgr2=0x7fc22c10d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 --2- 172.21.15.33:0/1517911858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1018a0 0x7fc22c10d5f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b0a0 tx=0x7fc21c00b3b0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 -- 172.21.15.33:0/1517911858 shutdown_connections 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 --2- 172.21.15.33:0/1517911858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1018a0 0x7fc22c10d5f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 --2- 172.21.15.33:0/1517911858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c1067b0 0x7fc22c106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 --2- 172.21.15.33:0/1517911858 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c100ee0 0x7fc22c101360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 -- 172.21.15.33:0/1517911858 >> 172.21.15.33:0/1517911858 conn(0x7fc22c076ce0 msgr2=0x7fc22c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 -- 172.21.15.33:0/1517911858 shutdown_connections 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.998+0000 7fc2339fa700 1 -- 172.21.15.33:0/1517911858 wait complete. 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 Processor -- start 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 -- start start 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c100ee0 0x7fc22c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c1018a0 0x7fc22c19de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1981c0 con 0x7fc22c1067b0 2024-01-25T16:06:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c198300 con 0x7fc22c100ee0 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc2339fa700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1984c0 con 0x7fc22c1018a0 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32938/0 (socket says 172.21.15.33:32938) 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231796700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c100ee0 0x7fc22c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 -- 172.21.15.33:0/3805655005 learned_addr learned my addr 172.21.15.33:0/3805655005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc230f95700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c1018a0 0x7fc22c19de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:58.228 INFO:teuthology.orchestra.run.smithi033.stdout:34359738450 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 -- 172.21.15.33:0/3805655005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c1018a0 msgr2=0x7fc22c19de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c1018a0 0x7fc22c19de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 -- 172.21.15.33:0/3805655005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c100ee0 msgr2=0x7fc22c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c100ee0 0x7fc22c19d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:57.999+0000 7fc231f97700 1 -- 172.21.15.33:0/3805655005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c009d00 con 0x7fc22c1067b0 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc231f97700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b070 tx=0x7fc21c007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:58.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00e040 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c198740 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc21c009440 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c01b450 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.000+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c1a49a0 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.001+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c0684a0 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.002+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fc21c01d480 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.002+0000 7fc22a7fc700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 0x7fc2140a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.002+0000 7fc231796700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 0x7fc2140a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.002+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc21c0cec50 con 0x7fc22c1067b0 2024-01-25T16:06:58.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.003+0000 7fc231796700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 0x7fc2140a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc22c105720 tx=0x7fc22400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.006+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc21c098000 con 0x7fc22c1067b0 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.218+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc22c1106b0 con 0x7fc22c1067b0 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.218+0000 7fc22a7fc700 1 -- 172.21.15.33:0/3805655005 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc21c09b620 con 0x7fc22c1067b0 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 msgr2=0x7fc2140a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 0x7fc2140a38c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc22c105720 tx=0x7fc22400a380 comp rx=0 tx=0).stop 2024-01-25T16:06:58.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 msgr2=0x7fc22c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b070 tx=0x7fc21c007870 comp rx=0 tx=0).stop 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 shutdown_connections 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc2140a1400 0x7fc2140a38c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc22c1067b0 0x7fc22c197b60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc22c100ee0 0x7fc22c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 --2- 172.21.15.33:0/3805655005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc22c1018a0 0x7fc22c19de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 >> 172.21.15.33:0/3805655005 conn(0x7fc22c076ce0 msgr2=0x7fc22c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 shutdown_connections 2024-01-25T16:06:58.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:58.220+0000 7fc2339fa700 1 -- 172.21.15.33:0/3805655005 wait complete. 2024-01-25T16:06:58.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738450 for osd.0 2024-01-25T16:06:58.812 DEBUG:teuthology.parallel:result is None 2024-01-25T16:06:59.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:59 smithi033 ceph-mon[31417]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:06:59.432 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:06:59 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3805655005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-25T16:06:59.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:59 smithi152 ceph-mon[36015]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:06:59.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:06:59 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3805655005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-25T16:06:59.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:59 smithi076 ceph-mon[35642]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:06:59.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:06:59 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3805655005' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 -- 172.21.15.33:0/864310057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c0ffdd0 msgr2=0x7f104c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 --2- 172.21.15.33:0/864310057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c0ffdd0 0x7f104c100250 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f103c009990 tx=0x7f103c009ca0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 -- 172.21.15.33:0/864310057 shutdown_connections 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 --2- 172.21.15.33:0/864310057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c100790 0x7f104c10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 --2- 172.21.15.33:0/864310057 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c106850 0x7f104c106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 --2- 172.21.15.33:0/864310057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c0ffdd0 0x7f104c100250 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 -- 172.21.15.33:0/864310057 >> 172.21.15.33:0/864310057 conn(0x7f104c076ce0 msgr2=0x7f104c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 -- 172.21.15.33:0/864310057 shutdown_connections 2024-01-25T16:06:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.756+0000 7f1053956700 1 -- 172.21.15.33:0/864310057 wait complete. 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 Processor -- start 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 -- start start 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c106850 0x7f104c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c198150 con 0x7f104c100790 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c198290 con 0x7f104c106850 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1053956700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c198450 con 0x7f104c0ffdd0 2024-01-25T16:06:59.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1050ef1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f10516f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1050ef1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53916/0 (socket says 172.21.15.33:53916) 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f10516f2700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:38154/0 (socket says 172.21.15.33:38154) 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1050ef1700 1 -- 172.21.15.33:0/730761223 learned_addr learned my addr 172.21.15.33:0/730761223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.757+0000 7f1051ef3700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c106850 0x7f104c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 -- 172.21.15.33:0/730761223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 msgr2=0x7f104c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 -- 172.21.15.33:0/730761223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c106850 msgr2=0x7f104c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c106850 0x7f104c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 -- 172.21.15.33:0/730761223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0095f0 con 0x7f104c100790 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f10516f2700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1050ef1700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f103c0098e0 tx=0x7f103c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c01d070 con 0x7f104c100790 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f103c022470 con 0x7f104c100790 2024-01-25T16:06:59.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c017650 con 0x7f104c100790 2024-01-25T16:06:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c1986d0 con 0x7f104c100790 2024-01-25T16:06:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.758+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c1a49d0 con 0x7f104c100790 2024-01-25T16:06:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.759+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104c109d70 con 0x7f104c100790 2024-01-25T16:06:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.761+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f103c0225e0 con 0x7f104c100790 2024-01-25T16:06:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.761+0000 7f10427fc700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 0x7f10380a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:06:59.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.762+0000 7f10516f2700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 0x7f10380a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:06:59.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.762+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f103c0ce010 con 0x7f104c100790 2024-01-25T16:06:59.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.762+0000 7f10516f2700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 0x7f10380a38c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1048005950 tx=0x7f104800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:06:59.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.769+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f103c0973c0 con 0x7f104c100790 2024-01-25T16:06:59.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.982+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f104c198fe0 con 0x7f104c100790 2024-01-25T16:06:59.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.983+0000 7f10427fc700 1 -- 172.21.15.33:0/730761223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f103c09a9e0 con 0x7f104c100790 2024-01-25T16:06:59.986 INFO:teuthology.orchestra.run.smithi033.stdout:257698037802 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 msgr2=0x7f10380a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 0x7f10380a38c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1048005950 tx=0x7f104800b410 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 msgr2=0x7f104c19de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f103c0098e0 tx=0x7f103c00f670 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 shutdown_connections 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f10380a1400 0x7f10380a38c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f104c106850 0x7f104c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f104c0ffdd0 0x7f104c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 --2- 172.21.15.33:0/730761223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f104c100790 0x7f104c19de70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 >> 172.21.15.33:0/730761223 conn(0x7f104c076ce0 msgr2=0x7f104c0fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:06:59.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 shutdown_connections 2024-01-25T16:06:59.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:06:59.985+0000 7f1053956700 1 -- 172.21.15.33:0/730761223 wait complete. 2024-01-25T16:07:00.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:00 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/730761223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-25T16:07:00.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:00 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.485 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:00 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/730761223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-25T16:07:00.485 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.485 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:00 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:00 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/730761223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-01-25T16:07:00.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:00 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:00.664 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037797 got 257698037802 for osd.8 2024-01-25T16:07:00.664 DEBUG:teuthology.parallel:result is None 2024-01-25T16:07:00.665 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-01-25T16:07:00.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph pg dump --format=json 2024-01-25T16:07:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:01 smithi033 ceph-mon[31417]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:01.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:01.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:01.315 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:01 smithi033 ceph-mon[31417]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:01 smithi152 ceph-mon[36015]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:01 smithi152 ceph-mon[36015]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:01.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:01 smithi076 ceph-mon[35642]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:01.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:01.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:01.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:01 smithi076 ceph-mon[35642]: from='mgr.14152 172.21.15.33:0/3414280909' entity='mgr.x' 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- 172.21.15.33:0/1696743560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c105590 msgr2=0x7fe45c105970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 --2- 172.21.15.33:0/1696743560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c105590 0x7fe45c105970 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009990 tx=0x7fe44c009ca0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- 172.21.15.33:0/1696743560 shutdown_connections 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 --2- 172.21.15.33:0/1696743560 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c101080 0x7fe45c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 --2- 172.21.15.33:0/1696743560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c105590 0x7fe45c105970 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 --2- 172.21.15.33:0/1696743560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 0x7fe45c100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- 172.21.15.33:0/1696743560 >> 172.21.15.33:0/1696743560 conn(0x7fe45c0776f0 msgr2=0x7fe45c077b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- 172.21.15.33:0/1696743560 shutdown_connections 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- 172.21.15.33:0/1696743560 wait complete. 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 Processor -- start 2024-01-25T16:07:02.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.483+0000 7fe461c76700 1 -- start start 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 0x7fe45c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c105590 0x7fe45c1a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1998e0 con 0x7fe45c101080 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1a40c0 con 0x7fe45c1006c0 2024-01-25T16:07:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1a4200 con 0x7fe45c105590 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53938/0 (socket says 172.21.15.33:53938) 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 -- 172.21.15.33:0/4440707 learned_addr learned my addr 172.21.15.33:0/4440707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 0x7fe45c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45bfff700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c105590 0x7fe45c1a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 -- 172.21.15.33:0/4440707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c105590 msgr2=0x7fe45c1a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c105590 0x7fe45c1a3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 -- 172.21.15.33:0/4440707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 msgr2=0x7fe45c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 0x7fe45c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 -- 172.21.15.33:0/4440707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0095f0 con 0x7fe45c101080 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe45affd700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d980 tx=0x7fe45000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500098d0 con 0x7fe45c101080 2024-01-25T16:07:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe450010460 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c1a44b0 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.484+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45000f620 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.485+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c1a4a00 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.486+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c108c00 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.487+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fe450010860 con 0x7fe45c101080 2024-01-25T16:07:02.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.487+0000 7fe458ff9700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 0x7fe4440a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:02.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.487+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 0x7fe4440a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:02.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.487+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4500cc9d0 con 0x7fe45c101080 2024-01-25T16:07:02.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.488+0000 7fe45b7fe700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 0x7fe4440a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b5c0 tx=0x7fe44c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:02.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.492+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe450095cf0 con 0x7fe45c101080 2024-01-25T16:07:02.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.689+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe45c110640 con 0x7fe4440a1480 2024-01-25T16:07:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.699+0000 7fe458ff9700 1 -- 172.21.15.33:0/4440707 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344395 (secure 0 0 0) 0x7fe45c110640 con 0x7fe4440a1480 2024-01-25T16:07:02.707 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 msgr2=0x7fe4440a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:02.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 0x7fe4440a3940 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00b5c0 tx=0x7fe44c005fd0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 msgr2=0x7fe45c199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe45000d980 tx=0x7fe45000dc90 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 shutdown_connections 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe4440a1480 0x7fe4440a3940 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe45c105590 0x7fe45c1a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe45c1006c0 0x7fe45c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 --2- 172.21.15.33:0/4440707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe45c101080 0x7fe45c199040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 >> 172.21.15.33:0/4440707 conn(0x7fe45c0776f0 msgr2=0x7fe45c0fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 shutdown_connections 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:02.702+0000 7fe461c76700 1 -- 172.21.15.33:0/4440707 wait complete. 2024-01-25T16:07:02.711 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-01-25T16:07:03.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:03 smithi033 ceph-mon[31417]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:03.374 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-01-25T16:07:02.117563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651233,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":410,"num_write":335,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366764,"kb_used_data":13392,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575060,"statfs":{"total":1151940427776,"available":1151564861440,"internally_reserved":0,"allocated":13713408,"data_stored":7470399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":17,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003000"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268274+0000","last_change":"2024-01-25T16:05:46.796305+0000","last_active":"2024-01-25T16:06:49.268274+0000","last_peered":"2024-01-25T16:06:49.268274+0000","last_clean":"2024-01-25T16:06:49.268274+0000","last_became_active":"2024-01-25T16:05:46.795989+0000","last_became_peered":"2024-01-25T16:05:46.795989+0000","last_unstale":"2024-01-25T16:06:49.268274+0000","last_undegraded":"2024-01-25T16:06:49.268274+0000","last_fullsized":"2024-01-25T16:06:49.268274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:45.764391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274441+0000","last_change":"2024-01-25T16:05:46.798335+0000","last_active":"2024-01-25T16:06:49.274441+0000","last_peered":"2024-01-25T16:06:49.274441+0000","last_clean":"2024-01-25T16:06:49.274441+0000","last_became_active":"2024-01-25T16:05:46.797743+0000","last_became_peered":"2024-01-25T16:05:46.797743+0000","last_unstale":"2024-01-25T16:06:49.274441+0000","last_undegraded":"2024-01-25T16:06:49.274441+0000","last_fullsized":"2024-01-25T16:06:49.274441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:06:01.335919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274046+0000","last_change":"2024-01-25T16:05:46.803057+0000","last_active":"2024-01-25T16:06:49.274046+0000","last_peered":"2024-01-25T16:06:49.274046+0000","last_clean":"2024-01-25T16:06:49.274046+0000","last_became_active":"2024-01-25T16:05:46.802706+0000","last_became_peered":"2024-01-25T16:05:46.802706+0000","last_unstale":"2024-01-25T16:06:49.274046+0000","last_undegraded":"2024-01-25T16:06:49.274046+0000","last_fullsized":"2024-01-25T16:06:49.274046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:27.159532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.691017+0000","last_change":"2024-01-25T16:05:46.796172+0000","last_active":"2024-01-25T16:06:49.691017+0000","last_peered":"2024-01-25T16:06:49.691017+0000","last_clean":"2024-01-25T16:06:49.691017+0000","last_became_active":"2024-01-25T16:05:46.795848+0000","last_became_peered":"2024-01-25T16:05:46.795848+0000","last_unstale":"2024-01-25T16:06:49.691017+0000","last_undegraded":"2024-01-25T16:06:49.691017+0000","last_fullsized":"2024-01-25T16:06:49.691017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:41:07.553896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274776+0000","last_change":"2024-01-25T16:05:46.794790+0000","last_active":"2024-01-25T16:06:49.274776+0000","last_peered":"2024-01-25T16:06:49.274776+0000","last_clean":"2024-01-25T16:06:49.274776+0000","last_became_active":"2024-01-25T16:05:46.794464+0000","last_became_peered":"2024-01-25T16:05:46.794464+0000","last_unstale":"2024-01-25T16:06:49.274776+0000","last_undegraded":"2024-01-25T16:06:49.274776+0000","last_fullsized":"2024-01-25T16:06:49.274776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:19:26.415959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.569416+0000","last_change":"2024-01-25T16:06:49.569416+0000","last_active":"2024-01-25T16:06:49.569416+0000","last_peered":"2024-01-25T16:06:49.569416+0000","last_clean":"2024-01-25T16:06:49.569416+0000","last_became_active":"2024-01-25T16:06:49.569006+0000","last_became_peered":"2024-01-25T16:06:49.569006+0000","last_unstale":"2024-01-25T16:06:49.569416+0000","last_undegraded":"2024-01-25T16:06:49.569416+0000","last_fullsized":"2024-01-25T16:06:49.569416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:38.469353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568423+0000","last_change":"2024-01-25T16:05:46.801432+0000","last_active":"2024-01-25T16:06:49.568423+0000","last_peered":"2024-01-25T16:06:49.568423+0000","last_clean":"2024-01-25T16:06:49.568423+0000","last_became_active":"2024-01-25T16:05:46.796673+0000","last_became_peered":"2024-01-25T16:05:46.796673+0000","last_unstale":"2024-01-25T16:06:49.568423+0000","last_undegraded":"2024-01-25T16:06:49.568423+0000","last_fullsized":"2024-01-25T16:06:49.568423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T04:02:37.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270781+0000","last_change":"2024-01-25T16:05:46.805885+0000","last_active":"2024-01-25T16:06:49.270781+0000","last_peered":"2024-01-25T16:06:49.270781+0000","last_clean":"2024-01-25T16:06:49.270781+0000","last_became_active":"2024-01-25T16:05:46.805359+0000","last_became_peered":"2024-01-25T16:05:46.805359+0000","last_unstale":"2024-01-25T16:06:49.270781+0000","last_undegraded":"2024-01-25T16:06:49.270781+0000","last_fullsized":"2024-01-25T16:06:49.270781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:44:54.641646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866917+0000","last_change":"2024-01-25T16:05:46.791944+0000","last_active":"2024-01-25T16:06:49.866917+0000","last_peered":"2024-01-25T16:06:49.866917+0000","last_clean":"2024-01-25T16:06:49.866917+0000","last_became_active":"2024-01-25T16:05:46.791637+0000","last_became_peered":"2024-01-25T16:05:46.791637+0000","last_unstale":"2024-01-25T16:06:49.866917+0000","last_undegraded":"2024-01-25T16:06:49.866917+0000","last_fullsized":"2024-01-25T16:06:49.866917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:54:20.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268025+0000","last_change":"2024-01-25T16:05:46.790961+0000","last_active":"2024-01-25T16:06:49.268025+0000","last_peered":"2024-01-25T16:06:49.268025+0000","last_clean":"2024-01-25T16:06:49.268025+0000","last_became_active":"2024-01-25T16:05:46.790664+0000","last_became_peered":"2024-01-25T16:05:46.790664+0000","last_unstale":"2024-01-25T16:06:49.268025+0000","last_undegraded":"2024-01-25T16:06:49.268025+0000","last_fullsized":"2024-01-25T16:06:49.268025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:24:57.615304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568162+0000","last_change":"2024-01-25T16:05:46.795205+0000","last_active":"2024-01-25T16:06:49.568162+0000","last_peered":"2024-01-25T16:06:49.568162+0000","last_clean":"2024-01-25T16:06:49.568162+0000","last_became_active":"2024-01-25T16:05:46.794894+0000","last_became_peered":"2024-01-25T16:05:46.794894+0000","last_unstale":"2024-01-25T16:06:49.568162+0000","last_undegraded":"2024-01-25T16:06:49.568162+0000","last_fullsized":"2024-01-25T16:06:49.568162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:57:37.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.690975+0000","last_change":"2024-01-25T16:05:46.795473+0000","last_active":"2024-01-25T16:06:49.690975+0000","last_peered":"2024-01-25T16:06:49.690975+0000","last_clean":"2024-01-25T16:06:49.690975+0000","last_became_active":"2024-01-25T16:05:46.795155+0000","last_became_peered":"2024-01-25T16:05:46.795155+0000","last_unstale":"2024-01-25T16:06:49.690975+0000","last_undegraded":"2024-01-25T16:06:49.690975+0000","last_fullsized":"2024-01-25T16:06:49.690975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:42:25.273806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143317+0000","last_change":"2024-01-25T16:05:46.792550+0000","last_active":"2024-01-25T16:06:50.143317+0000","last_peered":"2024-01-25T16:06:50.143317+0000","last_clean":"2024-01-25T16:06:50.143317+0000","last_became_active":"2024-01-25T16:05:46.792147+0000","last_became_peered":"2024-01-25T16:05:46.792147+0000","last_unstale":"2024-01-25T16:06:50.143317+0000","last_undegraded":"2024-01-25T16:06:50.143317+0000","last_fullsized":"2024-01-25T16:06:50.143317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:05:08.117739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273636+0000","last_change":"2024-01-25T16:05:46.808812+0000","last_active":"2024-01-25T16:06:49.273636+0000","last_peered":"2024-01-25T16:06:49.273636+0000","last_clean":"2024-01-25T16:06:49.273636+0000","last_became_active":"2024-01-25T16:05:46.808077+0000","last_became_peered":"2024-01-25T16:05:46.808077+0000","last_unstale":"2024-01-25T16:06:49.273636+0000","last_undegraded":"2024-01-25T16:06:49.273636+0000","last_fullsized":"2024-01-25T16:06:49.273636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:56:38.329223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273636+0000","last_change":"2024-01-25T16:05:46.796315+0000","last_active":"2024-01-25T16:06:49.273636+0000","last_peered":"2024-01-25T16:06:49.273636+0000","last_clean":"2024-01-25T16:06:49.273636+0000","last_became_active":"2024-01-25T16:05:46.795998+0000","last_became_peered":"2024-01-25T16:05:46.795998+0000","last_unstale":"2024-01-25T16:06:49.273636+0000","last_undegraded":"2024-01-25T16:06:49.273636+0000","last_fullsized":"2024-01-25T16:06:49.273636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:44:39.566605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866656+0000","last_change":"2024-01-25T16:05:46.798086+0000","last_active":"2024-01-25T16:06:49.866656+0000","last_peered":"2024-01-25T16:06:49.866656+0000","last_clean":"2024-01-25T16:06:49.866656+0000","last_became_active":"2024-01-25T16:05:46.797424+0000","last_became_peered":"2024-01-25T16:05:46.797424+0000","last_unstale":"2024-01-25T16:06:49.866656+0000","last_undegraded":"2024-01-25T16:06:49.866656+0000","last_fullsized":"2024-01-25T16:06:49.866656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:43:04.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268488+0000","last_change":"2024-01-25T16:05:46.799058+0000","last_active":"2024-01-25T16:06:49.268488+0000","last_peered":"2024-01-25T16:06:49.268488+0000","last_clean":"2024-01-25T16:06:49.268488+0000","last_became_active":"2024-01-25T16:05:46.798708+0000","last_became_peered":"2024-01-25T16:05:46.798708+0000","last_unstale":"2024-01-25T16:06:49.268488+0000","last_undegraded":"2024-01-25T16:06:49.268488+0000","last_fullsized":"2024-01-25T16:06:49.268488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:04:48.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272560+0000","last_change":"2024-01-25T16:05:46.791726+0000","last_active":"2024-01-25T16:06:49.272560+0000","last_peered":"2024-01-25T16:06:49.272560+0000","last_clean":"2024-01-25T16:06:49.272560+0000","last_became_active":"2024-01-25T16:05:46.791301+0000","last_became_peered":"2024-01-25T16:05:46.791301+0000","last_unstale":"2024-01-25T16:06:49.272560+0000","last_undegraded":"2024-01-25T16:06:49.272560+0000","last_fullsized":"2024-01-25T16:06:49.272560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:43:25.973750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270722+0000","last_change":"2024-01-25T16:05:46.795523+0000","last_active":"2024-01-25T16:06:49.270722+0000","last_peered":"2024-01-25T16:06:49.270722+0000","last_clean":"2024-01-25T16:06:49.270722+0000","last_became_active":"2024-01-25T16:05:46.794965+0000","last_became_peered":"2024-01-25T16:05:46.794965+0000","last_unstale":"2024-01-25T16:06:49.270722+0000","last_undegraded":"2024-01-25T16:06:49.270722+0000","last_fullsized":"2024-01-25T16:06:49.270722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:13:33.793210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272570+0000","last_change":"2024-01-25T16:05:45.764981+0000","last_active":"2024-01-25T16:06:49.272570+0000","last_peered":"2024-01-25T16:06:49.272570+0000","last_clean":"2024-01-25T16:06:49.272570+0000","last_became_active":"2024-01-25T16:05:45.764616+0000","last_became_peered":"2024-01-25T16:05:45.764616+0000","last_unstale":"2024-01-25T16:06:49.272570+0000","last_undegraded":"2024-01-25T16:06:49.272570+0000","last_fullsized":"2024-01-25T16:06:49.272570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:26:19.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270713+0000","last_change":"2024-01-25T16:05:46.800786+0000","last_active":"2024-01-25T16:06:49.270713+0000","last_peered":"2024-01-25T16:06:49.270713+0000","last_clean":"2024-01-25T16:06:49.270713+0000","last_became_active":"2024-01-25T16:05:46.799666+0000","last_became_peered":"2024-01-25T16:05:46.799666+0000","last_unstale":"2024-01-25T16:06:49.270713+0000","last_undegraded":"2024-01-25T16:06:49.270713+0000","last_fullsized":"2024-01-25T16:06:49.270713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:49:03.942394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271859+0000","last_change":"2024-01-25T16:05:45.773627+0000","last_active":"2024-01-25T16:06:49.271859+0000","last_peered":"2024-01-25T16:06:49.271859+0000","last_clean":"2024-01-25T16:06:49.271859+0000","last_became_active":"2024-01-25T16:05:45.773383+0000","last_became_peered":"2024-01-25T16:05:45.773383+0000","last_unstale":"2024-01-25T16:06:49.271859+0000","last_undegraded":"2024-01-25T16:06:49.271859+0000","last_fullsized":"2024-01-25T16:06:49.271859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:14:22.536001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274232+0000","last_change":"2024-01-25T16:05:46.801791+0000","last_active":"2024-01-25T16:06:49.274232+0000","last_peered":"2024-01-25T16:06:49.274232+0000","last_clean":"2024-01-25T16:06:49.274232+0000","last_became_active":"2024-01-25T16:05:46.801487+0000","last_became_peered":"2024-01-25T16:05:46.801487+0000","last_unstale":"2024-01-25T16:06:49.274232+0000","last_undegraded":"2024-01-25T16:06:49.274232+0000","last_fullsized":"2024-01-25T16:06:49.274232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:45:07.874907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275321+0000","last_change":"2024-01-25T16:05:45.774708+0000","last_active":"2024-01-25T16:06:49.275321+0000","last_peered":"2024-01-25T16:06:49.275321+0000","last_clean":"2024-01-25T16:06:49.275321+0000","last_became_active":"2024-01-25T16:05:45.774401+0000","last_became_peered":"2024-01-25T16:05:45.774401+0000","last_unstale":"2024-01-25T16:06:49.275321+0000","last_undegraded":"2024-01-25T16:06:49.275321+0000","last_fullsized":"2024-01-25T16:06:49.275321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:01:46.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866442+0000","last_change":"2024-01-25T16:05:46.794386+0000","last_active":"2024-01-25T16:06:49.866442+0000","last_peered":"2024-01-25T16:06:49.866442+0000","last_clean":"2024-01-25T16:06:49.866442+0000","last_became_active":"2024-01-25T16:05:46.793909+0000","last_became_peered":"2024-01-25T16:05:46.793909+0000","last_unstale":"2024-01-25T16:06:49.866442+0000","last_undegraded":"2024-01-25T16:06:49.866442+0000","last_fullsized":"2024-01-25T16:06:49.866442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:10:24.736904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275004+0000","last_change":"2024-01-25T16:05:45.767804+0000","last_active":"2024-01-25T16:06:49.275004+0000","last_peered":"2024-01-25T16:06:49.275004+0000","last_clean":"2024-01-25T16:06:49.275004+0000","last_became_active":"2024-01-25T16:05:45.767491+0000","last_became_peered":"2024-01-25T16:05:45.767491+0000","last_unstale":"2024-01-25T16:06:49.275004+0000","last_undegraded":"2024-01-25T16:06:49.275004+0000","last_fullsized":"2024-01-25T16:06:49.275004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:20:31.331950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556631+0000","last_change":"2024-01-25T16:05:46.803061+0000","last_active":"2024-01-25T16:06:49.556631+0000","last_peered":"2024-01-25T16:06:49.556631+0000","last_clean":"2024-01-25T16:06:49.556631+0000","last_became_active":"2024-01-25T16:05:46.802756+0000","last_became_peered":"2024-01-25T16:05:46.802756+0000","last_unstale":"2024-01-25T16:06:49.556631+0000","last_undegraded":"2024-01-25T16:06:49.556631+0000","last_fullsized":"2024-01-25T16:06:49.556631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:31:53.107217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.159128+0000","last_change":"2024-01-25T16:05:45.764304+0000","last_active":"2024-01-25T16:07:00.159128+0000","last_peered":"2024-01-25T16:07:00.159128+0000","last_clean":"2024-01-25T16:07:00.159128+0000","last_became_active":"2024-01-25T16:05:45.763977+0000","last_became_peered":"2024-01-25T16:05:45.763977+0000","last_unstale":"2024-01-25T16:07:00.159128+0000","last_undegraded":"2024-01-25T16:07:00.159128+0000","last_fullsized":"2024-01-25T16:07:00.159128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:16:07.031015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274510+0000","last_change":"2024-01-25T16:05:46.802631+0000","last_active":"2024-01-25T16:06:49.274510+0000","last_peered":"2024-01-25T16:06:49.274510+0000","last_clean":"2024-01-25T16:06:49.274510+0000","last_became_active":"2024-01-25T16:05:46.802305+0000","last_became_peered":"2024-01-25T16:05:46.802305+0000","last_unstale":"2024-01-25T16:06:49.274510+0000","last_undegraded":"2024-01-25T16:06:49.274510+0000","last_fullsized":"2024-01-25T16:06:49.274510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:26:59.388515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267678+0000","last_change":"2024-01-25T16:05:45.762541+0000","last_active":"2024-01-25T16:06:49.267678+0000","last_peered":"2024-01-25T16:06:49.267678+0000","last_clean":"2024-01-25T16:06:49.267678+0000","last_became_active":"2024-01-25T16:05:45.762238+0000","last_became_peered":"2024-01-25T16:05:45.762238+0000","last_unstale":"2024-01-25T16:06:49.267678+0000","last_undegraded":"2024-01-25T16:06:49.267678+0000","last_fullsized":"2024-01-25T16:06:49.267678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:56:41.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556500+0000","last_change":"2024-01-25T16:05:46.794741+0000","last_active":"2024-01-25T16:06:49.556500+0000","last_peered":"2024-01-25T16:06:49.556500+0000","last_clean":"2024-01-25T16:06:49.556500+0000","last_became_active":"2024-01-25T16:05:46.794294+0000","last_became_peered":"2024-01-25T16:05:46.794294+0000","last_unstale":"2024-01-25T16:06:49.556500+0000","last_undegraded":"2024-01-25T16:06:49.556500+0000","last_fullsized":"2024-01-25T16:06:49.556500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:43:00.876478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.142995+0000","last_change":"2024-01-25T16:05:45.766535+0000","last_active":"2024-01-25T16:06:50.142995+0000","last_peered":"2024-01-25T16:06:50.142995+0000","last_clean":"2024-01-25T16:06:50.142995+0000","last_became_active":"2024-01-25T16:05:45.765634+0000","last_became_peered":"2024-01-25T16:05:45.765634+0000","last_unstale":"2024-01-25T16:06:50.142995+0000","last_undegraded":"2024-01-25T16:06:50.142995+0000","last_fullsized":"2024-01-25T16:06:50.142995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:43:31.723438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270904+0000","last_change":"2024-01-25T16:05:46.793549+0000","last_active":"2024-01-25T16:06:49.270904+0000","last_peered":"2024-01-25T16:06:49.270904+0000","last_clean":"2024-01-25T16:06:49.270904+0000","last_became_active":"2024-01-25T16:05:46.793217+0000","last_became_peered":"2024-01-25T16:05:46.793217+0000","last_unstale":"2024-01-25T16:06:49.270904+0000","last_undegraded":"2024-01-25T16:06:49.270904+0000","last_fullsized":"2024-01-25T16:06:49.270904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:17:34.531729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556305+0000","last_change":"2024-01-25T16:05:45.768755+0000","last_active":"2024-01-25T16:06:49.556305+0000","last_peered":"2024-01-25T16:06:49.556305+0000","last_clean":"2024-01-25T16:06:49.556305+0000","last_became_active":"2024-01-25T16:05:45.768457+0000","last_became_peered":"2024-01-25T16:05:45.768457+0000","last_unstale":"2024-01-25T16:06:49.556305+0000","last_undegraded":"2024-01-25T16:06:49.556305+0000","last_fullsized":"2024-01-25T16:06:49.556305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:37:46.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270589+0000","last_change":"2024-01-25T16:05:46.792590+0000","last_active":"2024-01-25T16:06:49.270589+0000","last_peered":"2024-01-25T16:06:49.270589+0000","last_clean":"2024-01-25T16:06:49.270589+0000","last_became_active":"2024-01-25T16:05:46.792246+0000","last_became_peered":"2024-01-25T16:05:46.792246+0000","last_unstale":"2024-01-25T16:06:49.270589+0000","last_undegraded":"2024-01-25T16:06:49.270589+0000","last_fullsized":"2024-01-25T16:06:49.270589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:11:31.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:59.157176+0000","last_change":"2024-01-25T16:05:45.765720+0000","last_active":"2024-01-25T16:06:59.157176+0000","last_peered":"2024-01-25T16:06:59.157176+0000","last_clean":"2024-01-25T16:06:59.157176+0000","last_became_active":"2024-01-25T16:05:45.765336+0000","last_became_peered":"2024-01-25T16:05:45.765336+0000","last_unstale":"2024-01-25T16:06:59.157176+0000","last_undegraded":"2024-01-25T16:06:59.157176+0000","last_fullsized":"2024-01-25T16:06:59.157176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:26.016352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866840+0000","last_change":"2024-01-25T16:05:46.795603+0000","last_active":"2024-01-25T16:06:49.866840+0000","last_peered":"2024-01-25T16:06:49.866840+0000","last_clean":"2024-01-25T16:06:49.866840+0000","last_became_active":"2024-01-25T16:05:46.794725+0000","last_became_peered":"2024-01-25T16:05:46.794725+0000","last_unstale":"2024-01-25T16:06:49.866840+0000","last_undegraded":"2024-01-25T16:06:49.866840+0000","last_fullsized":"2024-01-25T16:06:49.866840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:08:28.823911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568169+0000","last_change":"2024-01-25T16:05:45.769249+0000","last_active":"2024-01-25T16:06:49.568169+0000","last_peered":"2024-01-25T16:06:49.568169+0000","last_clean":"2024-01-25T16:06:49.568169+0000","last_became_active":"2024-01-25T16:05:45.768939+0000","last_became_peered":"2024-01-25T16:05:45.768939+0000","last_unstale":"2024-01-25T16:06:49.568169+0000","last_undegraded":"2024-01-25T16:06:49.568169+0000","last_fullsized":"2024-01-25T16:06:49.568169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:32:46.727035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.557905+0000","last_change":"2024-01-25T16:06:49.557905+0000","last_active":"2024-01-25T16:06:49.557905+0000","last_peered":"2024-01-25T16:06:49.557905+0000","last_clean":"2024-01-25T16:06:49.557905+0000","last_became_active":"2024-01-25T16:06:49.557559+0000","last_became_peered":"2024-01-25T16:06:49.557559+0000","last_unstale":"2024-01-25T16:06:49.557905+0000","last_undegraded":"2024-01-25T16:06:49.557905+0000","last_fullsized":"2024-01-25T16:06:49.557905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:25:50.375970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556885+0000","last_change":"2024-01-25T16:05:45.758681+0000","last_active":"2024-01-25T16:06:49.556885+0000","last_peered":"2024-01-25T16:06:49.556885+0000","last_clean":"2024-01-25T16:06:49.556885+0000","last_became_active":"2024-01-25T16:05:45.758394+0000","last_became_peered":"2024-01-25T16:05:45.758394+0000","last_unstale":"2024-01-25T16:06:49.556885+0000","last_undegraded":"2024-01-25T16:06:49.556885+0000","last_fullsized":"2024-01-25T16:06:49.556885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:55:50.498848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866562+0000","last_change":"2024-01-25T16:05:46.794286+0000","last_active":"2024-01-25T16:06:49.866562+0000","last_peered":"2024-01-25T16:06:49.866562+0000","last_clean":"2024-01-25T16:06:49.866562+0000","last_became_active":"2024-01-25T16:05:46.793949+0000","last_became_peered":"2024-01-25T16:05:46.793949+0000","last_unstale":"2024-01-25T16:06:49.866562+0000","last_undegraded":"2024-01-25T16:06:49.866562+0000","last_fullsized":"2024-01-25T16:06:49.866562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:47:02.196054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.158530+0000","last_change":"2024-01-25T16:05:45.761374+0000","last_active":"2024-01-25T16:07:00.158530+0000","last_peered":"2024-01-25T16:07:00.158530+0000","last_clean":"2024-01-25T16:07:00.158530+0000","last_became_active":"2024-01-25T16:05:45.761060+0000","last_became_peered":"2024-01-25T16:05:45.761060+0000","last_unstale":"2024-01-25T16:07:00.158530+0000","last_undegraded":"2024-01-25T16:07:00.158530+0000","last_fullsized":"2024-01-25T16:07:00.158530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:56:40.591127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56631,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":149,"num_read_kb":129,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274945+0000","last_change":"2024-01-25T16:05:46.799163+0000","last_active":"2024-01-25T16:06:49.274945+0000","last_peered":"2024-01-25T16:06:49.274945+0000","last_clean":"2024-01-25T16:06:49.274945+0000","last_became_active":"2024-01-25T16:05:46.798771+0000","last_became_peered":"2024-01-25T16:05:46.798771+0000","last_unstale":"2024-01-25T16:06:49.274945+0000","last_undegraded":"2024-01-25T16:06:49.274945+0000","last_fullsized":"2024-01-25T16:06:49.274945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:30:23.693300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.282719+0000","last_change":"2024-01-25T16:06:49.282204+0000","last_active":"2024-01-25T16:06:49.282719+0000","last_peered":"2024-01-25T16:06:49.282719+0000","last_clean":"2024-01-25T16:06:49.282719+0000","last_became_active":"2024-01-25T16:06:49.273746+0000","last_became_peered":"2024-01-25T16:06:49.273746+0000","last_unstale":"2024-01-25T16:06:49.282719+0000","last_undegraded":"2024-01-25T16:06:49.282719+0000","last_fullsized":"2024-01-25T16:06:49.282719+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:49.400315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143250+0000","last_change":"2024-01-25T16:05:46.792576+0000","last_active":"2024-01-25T16:06:50.143250+0000","last_peered":"2024-01-25T16:06:50.143250+0000","last_clean":"2024-01-25T16:06:50.143250+0000","last_became_active":"2024-01-25T16:05:46.792220+0000","last_became_peered":"2024-01-25T16:05:46.792220+0000","last_unstale":"2024-01-25T16:06:50.143250+0000","last_undegraded":"2024-01-25T16:06:50.143250+0000","last_fullsized":"2024-01-25T16:06:50.143250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:49:31.388289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556417+0000","last_change":"2024-01-25T16:05:45.767265+0000","last_active":"2024-01-25T16:06:49.556417+0000","last_peered":"2024-01-25T16:06:49.556417+0000","last_clean":"2024-01-25T16:06:49.556417+0000","last_became_active":"2024-01-25T16:05:45.766877+0000","last_became_peered":"2024-01-25T16:05:45.766877+0000","last_unstale":"2024-01-25T16:06:49.556417+0000","last_undegraded":"2024-01-25T16:06:49.556417+0000","last_fullsized":"2024-01-25T16:06:49.556417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:47:54.705160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568171+0000","last_change":"2024-01-25T16:05:46.801811+0000","last_active":"2024-01-25T16:06:49.568171+0000","last_peered":"2024-01-25T16:06:49.568171+0000","last_clean":"2024-01-25T16:06:49.568171+0000","last_became_active":"2024-01-25T16:05:46.799546+0000","last_became_peered":"2024-01-25T16:05:46.799546+0000","last_unstale":"2024-01-25T16:06:49.568171+0000","last_undegraded":"2024-01-25T16:06:49.568171+0000","last_fullsized":"2024-01-25T16:06:49.568171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:18:52.900456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.158290+0000","last_change":"2024-01-25T16:05:45.769514+0000","last_active":"2024-01-25T16:07:00.158290+0000","last_peered":"2024-01-25T16:07:00.158290+0000","last_clean":"2024-01-25T16:07:00.158290+0000","last_became_active":"2024-01-25T16:05:45.768497+0000","last_became_peered":"2024-01-25T16:05:45.768497+0000","last_unstale":"2024-01-25T16:07:00.158290+0000","last_undegraded":"2024-01-25T16:07:00.158290+0000","last_fullsized":"2024-01-25T16:07:00.158290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:37.418948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567772+0000","last_change":"2024-01-25T16:05:46.798502+0000","last_active":"2024-01-25T16:06:49.567772+0000","last_peered":"2024-01-25T16:06:49.567772+0000","last_clean":"2024-01-25T16:06:49.567772+0000","last_became_active":"2024-01-25T16:05:46.798203+0000","last_became_peered":"2024-01-25T16:05:46.798203+0000","last_unstale":"2024-01-25T16:06:49.567772+0000","last_undegraded":"2024-01-25T16:06:49.567772+0000","last_fullsized":"2024-01-25T16:06:49.567772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:31:53.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274509+0000","last_change":"2024-01-25T16:05:45.764827+0000","last_active":"2024-01-25T16:06:49.274509+0000","last_peered":"2024-01-25T16:06:49.274509+0000","last_clean":"2024-01-25T16:06:49.274509+0000","last_became_active":"2024-01-25T16:05:45.764466+0000","last_became_peered":"2024-01-25T16:05:45.764466+0000","last_unstale":"2024-01-25T16:06:49.274509+0000","last_undegraded":"2024-01-25T16:06:49.274509+0000","last_fullsized":"2024-01-25T16:06:49.274509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:57:17.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556199+0000","last_change":"2024-01-25T16:05:46.803960+0000","last_active":"2024-01-25T16:06:49.556199+0000","last_peered":"2024-01-25T16:06:49.556199+0000","last_clean":"2024-01-25T16:06:49.556199+0000","last_became_active":"2024-01-25T16:05:46.803246+0000","last_became_peered":"2024-01-25T16:05:46.803246+0000","last_unstale":"2024-01-25T16:06:49.556199+0000","last_undegraded":"2024-01-25T16:06:49.556199+0000","last_fullsized":"2024-01-25T16:06:49.556199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:42.354813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275598+0000","last_change":"2024-01-25T16:05:45.765952+0000","last_active":"2024-01-25T16:06:49.275598+0000","last_peered":"2024-01-25T16:06:49.275598+0000","last_clean":"2024-01-25T16:06:49.275598+0000","last_became_active":"2024-01-25T16:05:45.765575+0000","last_became_peered":"2024-01-25T16:05:45.765575+0000","last_unstale":"2024-01-25T16:06:49.275598+0000","last_undegraded":"2024-01-25T16:06:49.275598+0000","last_fullsized":"2024-01-25T16:06:49.275598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:12:07.423245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.558287+0000","last_change":"2024-01-25T16:06:49.558287+0000","last_active":"2024-01-25T16:06:49.558287+0000","last_peered":"2024-01-25T16:06:49.558287+0000","last_clean":"2024-01-25T16:06:49.558287+0000","last_became_active":"2024-01-25T16:06:49.557763+0000","last_became_peered":"2024-01-25T16:06:49.557763+0000","last_unstale":"2024-01-25T16:06:49.558287+0000","last_undegraded":"2024-01-25T16:06:49.558287+0000","last_fullsized":"2024-01-25T16:06:49.558287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:28.662830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271711+0000","last_change":"2024-01-25T16:05:45.765934+0000","last_active":"2024-01-25T16:06:49.271711+0000","last_peered":"2024-01-25T16:06:49.271711+0000","last_clean":"2024-01-25T16:06:49.271711+0000","last_became_active":"2024-01-25T16:05:45.765583+0000","last_became_peered":"2024-01-25T16:05:45.765583+0000","last_unstale":"2024-01-25T16:06:49.271711+0000","last_undegraded":"2024-01-25T16:06:49.271711+0000","last_fullsized":"2024-01-25T16:06:49.271711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:42:45.814218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267932+0000","last_change":"2024-01-25T16:05:46.794622+0000","last_active":"2024-01-25T16:06:49.267932+0000","last_peered":"2024-01-25T16:06:49.267932+0000","last_clean":"2024-01-25T16:06:49.267932+0000","last_became_active":"2024-01-25T16:05:46.794304+0000","last_became_peered":"2024-01-25T16:05:46.794304+0000","last_unstale":"2024-01-25T16:06:49.267932+0000","last_undegraded":"2024-01-25T16:06:49.267932+0000","last_fullsized":"2024-01-25T16:06:49.267932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:48:31.605501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568387+0000","last_change":"2024-01-25T16:05:45.772412+0000","last_active":"2024-01-25T16:06:49.568387+0000","last_peered":"2024-01-25T16:06:49.568387+0000","last_clean":"2024-01-25T16:06:49.568387+0000","last_became_active":"2024-01-25T16:05:45.772074+0000","last_became_peered":"2024-01-25T16:05:45.772074+0000","last_unstale":"2024-01-25T16:06:49.568387+0000","last_undegraded":"2024-01-25T16:06:49.568387+0000","last_fullsized":"2024-01-25T16:06:49.568387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:42:46.858459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267946+0000","last_change":"2024-01-25T16:05:46.800980+0000","last_active":"2024-01-25T16:06:49.267946+0000","last_peered":"2024-01-25T16:06:49.267946+0000","last_clean":"2024-01-25T16:06:49.267946+0000","last_became_active":"2024-01-25T16:05:46.800689+0000","last_became_peered":"2024-01-25T16:05:46.800689+0000","last_unstale":"2024-01-25T16:06:49.267946+0000","last_undegraded":"2024-01-25T16:06:49.267946+0000","last_fullsized":"2024-01-25T16:06:49.267946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:15:05.197587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556613+0000","last_change":"2024-01-25T16:05:46.802971+0000","last_active":"2024-01-25T16:06:49.556613+0000","last_peered":"2024-01-25T16:06:49.556613+0000","last_clean":"2024-01-25T16:06:49.556613+0000","last_became_active":"2024-01-25T16:05:46.802669+0000","last_became_peered":"2024-01-25T16:05:46.802669+0000","last_unstale":"2024-01-25T16:06:49.556613+0000","last_undegraded":"2024-01-25T16:06:49.556613+0000","last_fullsized":"2024-01-25T16:06:49.556613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:23:01.261255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.157687+0000","last_change":"2024-01-25T16:05:45.763052+0000","last_active":"2024-01-25T16:07:00.157687+0000","last_peered":"2024-01-25T16:07:00.157687+0000","last_clean":"2024-01-25T16:07:00.157687+0000","last_became_active":"2024-01-25T16:05:45.762745+0000","last_became_peered":"2024-01-25T16:05:45.762745+0000","last_unstale":"2024-01-25T16:07:00.157687+0000","last_undegraded":"2024-01-25T16:07:00.157687+0000","last_fullsized":"2024-01-25T16:07:00.157687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:04.783033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271971+0000","last_change":"2024-01-25T16:05:46.799092+0000","last_active":"2024-01-25T16:06:49.271971+0000","last_peered":"2024-01-25T16:06:49.271971+0000","last_clean":"2024-01-25T16:06:49.271971+0000","last_became_active":"2024-01-25T16:05:46.798693+0000","last_became_peered":"2024-01-25T16:05:46.798693+0000","last_unstale":"2024-01-25T16:06:49.271971+0000","last_undegraded":"2024-01-25T16:06:49.271971+0000","last_fullsized":"2024-01-25T16:06:49.271971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:26:29.504645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.690896+0000","last_change":"2024-01-25T16:05:45.760805+0000","last_active":"2024-01-25T16:06:49.690896+0000","last_peered":"2024-01-25T16:06:49.690896+0000","last_clean":"2024-01-25T16:06:49.690896+0000","last_became_active":"2024-01-25T16:05:45.760420+0000","last_became_peered":"2024-01-25T16:05:45.760420+0000","last_unstale":"2024-01-25T16:06:49.690896+0000","last_undegraded":"2024-01-25T16:06:49.690896+0000","last_fullsized":"2024-01-25T16:06:49.690896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:14:16.150271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567893+0000","last_change":"2024-01-25T16:04:34.048543+0000","last_active":"2024-01-25T16:06:49.567893+0000","last_peered":"2024-01-25T16:06:49.567893+0000","last_clean":"2024-01-25T16:06:49.567893+0000","last_became_active":"2024-01-25T16:04:34.048051+0000","last_became_peered":"2024-01-25T16:04:34.048051+0000","last_unstale":"2024-01-25T16:06:49.567893+0000","last_undegraded":"2024-01-25T16:06:49.567893+0000","last_fullsized":"2024-01-25T16:06:49.567893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_clean_scrub_stamp":"2024-01-25T16:01:16.962963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:34:03.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143304+0000","last_change":"2024-01-25T16:05:46.793372+0000","last_active":"2024-01-25T16:06:50.143304+0000","last_peered":"2024-01-25T16:06:50.143304+0000","last_clean":"2024-01-25T16:06:50.143304+0000","last_became_active":"2024-01-25T16:05:46.792312+0000","last_became_peered":"2024-01-25T16:05:46.792312+0000","last_unstale":"2024-01-25T16:06:50.143304+0000","last_undegraded":"2024-01-25T16:06:50.143304+0000","last_fullsized":"2024-01-25T16:06:50.143304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:34:12.261247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567957+0000","last_change":"2024-01-25T16:05:45.776106+0000","last_active":"2024-01-25T16:06:49.567957+0000","last_peered":"2024-01-25T16:06:49.567957+0000","last_clean":"2024-01-25T16:06:49.567957+0000","last_became_active":"2024-01-25T16:05:45.775880+0000","last_became_peered":"2024-01-25T16:05:45.775880+0000","last_unstale":"2024-01-25T16:06:49.567957+0000","last_undegraded":"2024-01-25T16:06:49.567957+0000","last_fullsized":"2024-01-25T16:06:49.567957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:30:00.130932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275103+0000","last_change":"2024-01-25T16:05:45.775465+0000","last_active":"2024-01-25T16:06:49.275103+0000","last_peered":"2024-01-25T16:06:49.275103+0000","last_clean":"2024-01-25T16:06:49.275103+0000","last_became_active":"2024-01-25T16:05:45.775068+0000","last_became_peered":"2024-01-25T16:05:45.775068+0000","last_unstale":"2024-01-25T16:06:49.275103+0000","last_undegraded":"2024-01-25T16:06:49.275103+0000","last_fullsized":"2024-01-25T16:06:49.275103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:29:27.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143263+0000","last_change":"2024-01-25T16:05:46.788033+0000","last_active":"2024-01-25T16:06:50.143263+0000","last_peered":"2024-01-25T16:06:50.143263+0000","last_clean":"2024-01-25T16:06:50.143263+0000","last_became_active":"2024-01-25T16:05:46.787700+0000","last_became_peered":"2024-01-25T16:05:46.787700+0000","last_unstale":"2024-01-25T16:06:50.143263+0000","last_undegraded":"2024-01-25T16:06:50.143263+0000","last_fullsized":"2024-01-25T16:06:50.143263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:41:02.923376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.286189+0000","last_change":"2024-01-25T16:06:49.286189+0000","last_active":"2024-01-25T16:06:49.286189+0000","last_peered":"2024-01-25T16:06:49.286189+0000","last_clean":"2024-01-25T16:06:49.286189+0000","last_became_active":"2024-01-25T16:06:49.274442+0000","last_became_peered":"2024-01-25T16:06:49.274442+0000","last_unstale":"2024-01-25T16:06:49.286189+0000","last_undegraded":"2024-01-25T16:06:49.286189+0000","last_fullsized":"2024-01-25T16:06:49.286189+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:34:14.971542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272127+0000","last_change":"2024-01-25T16:05:46.810382+0000","last_active":"2024-01-25T16:06:49.272127+0000","last_peered":"2024-01-25T16:06:49.272127+0000","last_clean":"2024-01-25T16:06:49.272127+0000","last_became_active":"2024-01-25T16:05:46.806972+0000","last_became_peered":"2024-01-25T16:05:46.806972+0000","last_unstale":"2024-01-25T16:06:49.272127+0000","last_undegraded":"2024-01-25T16:06:49.272127+0000","last_fullsized":"2024-01-25T16:06:49.272127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:54:44.104250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.691434+0000","last_change":"2024-01-25T16:05:45.767147+0000","last_active":"2024-01-25T16:06:49.691434+0000","last_peered":"2024-01-25T16:06:49.691434+0000","last_clean":"2024-01-25T16:06:49.691434+0000","last_became_active":"2024-01-25T16:05:45.766811+0000","last_became_peered":"2024-01-25T16:05:45.766811+0000","last_unstale":"2024-01-25T16:06:49.691434+0000","last_undegraded":"2024-01-25T16:06:49.691434+0000","last_fullsized":"2024-01-25T16:06:49.691434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:51:59.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.144682+0000","last_change":"2024-01-25T16:06:50.144682+0000","last_active":"2024-01-25T16:06:50.144682+0000","last_peered":"2024-01-25T16:06:50.144682+0000","last_clean":"2024-01-25T16:06:50.144682+0000","last_became_active":"2024-01-25T16:06:50.144301+0000","last_became_peered":"2024-01-25T16:06:50.144301+0000","last_unstale":"2024-01-25T16:06:50.144682+0000","last_undegraded":"2024-01-25T16:06:50.144682+0000","last_fullsized":"2024-01-25T16:06:50.144682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:14:24.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866249+0000","last_change":"2024-01-25T16:05:45.761456+0000","last_active":"2024-01-25T16:06:49.866249+0000","last_peered":"2024-01-25T16:06:49.866249+0000","last_clean":"2024-01-25T16:06:49.866249+0000","last_became_active":"2024-01-25T16:05:45.761127+0000","last_became_peered":"2024-01-25T16:05:45.761127+0000","last_unstale":"2024-01-25T16:06:49.866249+0000","last_undegraded":"2024-01-25T16:06:49.866249+0000","last_fullsized":"2024-01-25T16:06:49.866249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:44:00.091230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274600+0000","last_change":"2024-01-25T16:05:46.792572+0000","last_active":"2024-01-25T16:06:49.274600+0000","last_peered":"2024-01-25T16:06:49.274600+0000","last_clean":"2024-01-25T16:06:49.274600+0000","last_became_active":"2024-01-25T16:05:46.792201+0000","last_became_peered":"2024-01-25T16:05:46.792201+0000","last_unstale":"2024-01-25T16:06:49.274600+0000","last_undegraded":"2024-01-25T16:06:49.274600+0000","last_fullsized":"2024-01-25T16:06:49.274600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:17:29.636840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271601+0000","last_change":"2024-01-25T16:05:45.770183+0000","last_active":"2024-01-25T16:06:49.271601+0000","last_peered":"2024-01-25T16:06:49.271601+0000","last_clean":"2024-01-25T16:06:49.271601+0000","last_became_active":"2024-01-25T16:05:45.769573+0000","last_became_peered":"2024-01-25T16:05:45.769573+0000","last_unstale":"2024-01-25T16:06:49.271601+0000","last_undegraded":"2024-01-25T16:06:49.271601+0000","last_fullsized":"2024-01-25T16:06:49.271601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:39.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267878+0000","last_change":"2024-01-25T16:05:46.795137+0000","last_active":"2024-01-25T16:06:49.267878+0000","last_peered":"2024-01-25T16:06:49.267878+0000","last_clean":"2024-01-25T16:06:49.267878+0000","last_became_active":"2024-01-25T16:05:46.794836+0000","last_became_peered":"2024-01-25T16:05:46.794836+0000","last_unstale":"2024-01-25T16:06:49.267878+0000","last_undegraded":"2024-01-25T16:06:49.267878+0000","last_fullsized":"2024-01-25T16:06:49.267878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:51:27.421203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270539+0000","last_change":"2024-01-25T16:05:45.765813+0000","last_active":"2024-01-25T16:06:49.270539+0000","last_peered":"2024-01-25T16:06:49.270539+0000","last_clean":"2024-01-25T16:06:49.270539+0000","last_became_active":"2024-01-25T16:05:45.765520+0000","last_became_peered":"2024-01-25T16:05:45.765520+0000","last_unstale":"2024-01-25T16:06:49.270539+0000","last_undegraded":"2024-01-25T16:06:49.270539+0000","last_fullsized":"2024-01-25T16:06:49.270539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:39:45.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272666+0000","last_change":"2024-01-25T16:05:46.805665+0000","last_active":"2024-01-25T16:06:49.272666+0000","last_peered":"2024-01-25T16:06:49.272666+0000","last_clean":"2024-01-25T16:06:49.272666+0000","last_became_active":"2024-01-25T16:05:46.804989+0000","last_became_peered":"2024-01-25T16:05:46.804989+0000","last_unstale":"2024-01-25T16:06:49.272666+0000","last_undegraded":"2024-01-25T16:06:49.272666+0000","last_fullsized":"2024-01-25T16:06:49.272666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:35:07.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274825+0000","last_change":"2024-01-25T16:05:45.769982+0000","last_active":"2024-01-25T16:06:49.274825+0000","last_peered":"2024-01-25T16:06:49.274825+0000","last_clean":"2024-01-25T16:06:49.274825+0000","last_became_active":"2024-01-25T16:05:45.769617+0000","last_became_peered":"2024-01-25T16:05:45.769617+0000","last_unstale":"2024-01-25T16:06:49.274825+0000","last_undegraded":"2024-01-25T16:06:49.274825+0000","last_fullsized":"2024-01-25T16:06:49.274825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:23.225258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568330+0000","last_change":"2024-01-25T16:05:46.793665+0000","last_active":"2024-01-25T16:06:49.568330+0000","last_peered":"2024-01-25T16:06:49.568330+0000","last_clean":"2024-01-25T16:06:49.568330+0000","last_became_active":"2024-01-25T16:05:46.793335+0000","last_became_peered":"2024-01-25T16:05:46.793335+0000","last_unstale":"2024-01-25T16:06:49.568330+0000","last_undegraded":"2024-01-25T16:06:49.568330+0000","last_fullsized":"2024-01-25T16:06:49.568330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:43:51.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271009+0000","last_change":"2024-01-25T16:05:45.768136+0000","last_active":"2024-01-25T16:06:49.271009+0000","last_peered":"2024-01-25T16:06:49.271009+0000","last_clean":"2024-01-25T16:06:49.271009+0000","last_became_active":"2024-01-25T16:05:45.767847+0000","last_became_peered":"2024-01-25T16:05:45.767847+0000","last_unstale":"2024-01-25T16:06:49.271009+0000","last_undegraded":"2024-01-25T16:06:49.271009+0000","last_fullsized":"2024-01-25T16:06:49.271009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:11:16.154786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568231+0000","last_change":"2024-01-25T16:05:46.794454+0000","last_active":"2024-01-25T16:06:49.568231+0000","last_peered":"2024-01-25T16:06:49.568231+0000","last_clean":"2024-01-25T16:06:49.568231+0000","last_became_active":"2024-01-25T16:05:46.794135+0000","last_became_peered":"2024-01-25T16:05:46.794135+0000","last_unstale":"2024-01-25T16:06:49.568231+0000","last_undegraded":"2024-01-25T16:06:49.568231+0000","last_fullsized":"2024-01-25T16:06:49.568231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:17:56.406013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556693+0000","last_change":"2024-01-25T16:05:45.763459+0000","last_active":"2024-01-25T16:06:49.556693+0000","last_peered":"2024-01-25T16:06:49.556693+0000","last_clean":"2024-01-25T16:06:49.556693+0000","last_became_active":"2024-01-25T16:05:45.763142+0000","last_became_peered":"2024-01-25T16:05:45.763142+0000","last_unstale":"2024-01-25T16:06:49.556693+0000","last_undegraded":"2024-01-25T16:06:49.556693+0000","last_fullsized":"2024-01-25T16:06:49.556693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:52:59.970813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568059+0000","last_change":"2024-01-25T16:05:46.797135+0000","last_active":"2024-01-25T16:06:49.568059+0000","last_peered":"2024-01-25T16:06:49.568059+0000","last_clean":"2024-01-25T16:06:49.568059+0000","last_became_active":"2024-01-25T16:05:46.796838+0000","last_became_peered":"2024-01-25T16:05:46.796838+0000","last_unstale":"2024-01-25T16:06:49.568059+0000","last_undegraded":"2024-01-25T16:06:49.568059+0000","last_fullsized":"2024-01-25T16:06:49.568059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:36:59.271257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143019+0000","last_change":"2024-01-25T16:05:45.762903+0000","last_active":"2024-01-25T16:06:50.143019+0000","last_peered":"2024-01-25T16:06:50.143019+0000","last_clean":"2024-01-25T16:06:50.143019+0000","last_became_active":"2024-01-25T16:05:45.762559+0000","last_became_peered":"2024-01-25T16:05:45.762559+0000","last_unstale":"2024-01-25T16:06:50.143019+0000","last_undegraded":"2024-01-25T16:06:50.143019+0000","last_fullsized":"2024-01-25T16:06:50.143019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:58:57.511914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568299+0000","last_change":"2024-01-25T16:05:46.792890+0000","last_active":"2024-01-25T16:06:49.568299+0000","last_peered":"2024-01-25T16:06:49.568299+0000","last_clean":"2024-01-25T16:06:49.568299+0000","last_became_active":"2024-01-25T16:05:46.792561+0000","last_became_peered":"2024-01-25T16:05:46.792561+0000","last_unstale":"2024-01-25T16:06:49.568299+0000","last_undegraded":"2024-01-25T16:06:49.568299+0000","last_fullsized":"2024-01-25T16:06:49.568299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:55:20.348743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272334+0000","last_change":"2024-01-25T16:06:49.272334+0000","last_active":"2024-01-25T16:06:49.272334+0000","last_peered":"2024-01-25T16:06:49.272334+0000","last_clean":"2024-01-25T16:06:49.272334+0000","last_became_active":"2024-01-25T16:06:49.271856+0000","last_became_peered":"2024-01-25T16:06:49.271856+0000","last_unstale":"2024-01-25T16:06:49.272334+0000","last_undegraded":"2024-01-25T16:06:49.272334+0000","last_fullsized":"2024-01-25T16:06:49.272334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:11:46.364112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.692112+0000","last_change":"2024-01-25T16:06:49.692112+0000","last_active":"2024-01-25T16:06:49.692112+0000","last_peered":"2024-01-25T16:06:49.692112+0000","last_clean":"2024-01-25T16:06:49.692112+0000","last_became_active":"2024-01-25T16:06:49.691777+0000","last_became_peered":"2024-01-25T16:06:49.691777+0000","last_unstale":"2024-01-25T16:06:49.692112+0000","last_undegraded":"2024-01-25T16:06:49.692112+0000","last_fullsized":"2024-01-25T16:06:49.692112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:02:14.205358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270501+0000","last_change":"2024-01-25T16:05:46.797036+0000","last_active":"2024-01-25T16:06:49.270501+0000","last_peered":"2024-01-25T16:06:49.270501+0000","last_clean":"2024-01-25T16:06:49.270501+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:06:49.270501+0000","last_undegraded":"2024-01-25T16:06:49.270501+0000","last_fullsized":"2024-01-25T16:06:49.270501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:06:00.095666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273783+0000","last_change":"2024-01-25T16:05:46.796367+0000","last_active":"2024-01-25T16:06:49.273783+0000","last_peered":"2024-01-25T16:06:49.273783+0000","last_clean":"2024-01-25T16:06:49.273783+0000","last_became_active":"2024-01-25T16:05:46.795390+0000","last_became_peered":"2024-01-25T16:05:46.795390+0000","last_unstale":"2024-01-25T16:06:49.273783+0000","last_undegraded":"2024-01-25T16:06:49.273783+0000","last_fullsized":"2024-01-25T16:06:49.273783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:35:41.524542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143160+0000","last_change":"2024-01-25T16:05:46.796127+0000","last_active":"2024-01-25T16:06:50.143160+0000","last_peered":"2024-01-25T16:06:50.143160+0000","last_clean":"2024-01-25T16:06:50.143160+0000","last_became_active":"2024-01-25T16:05:46.795667+0000","last_became_peered":"2024-01-25T16:05:46.795667+0000","last_unstale":"2024-01-25T16:06:50.143160+0000","last_undegraded":"2024-01-25T16:06:50.143160+0000","last_fullsized":"2024-01-25T16:06:50.143160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:00:44.411676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268030+0000","last_change":"2024-01-25T16:05:46.796491+0000","last_active":"2024-01-25T16:06:49.268030+0000","last_peered":"2024-01-25T16:06:49.268030+0000","last_clean":"2024-01-25T16:06:49.268030+0000","last_became_active":"2024-01-25T16:05:46.796201+0000","last_became_peered":"2024-01-25T16:05:46.796201+0000","last_unstale":"2024-01-25T16:06:49.268030+0000","last_undegraded":"2024-01-25T16:06:49.268030+0000","last_fullsized":"2024-01-25T16:06:49.268030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:39:42.396263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271125+0000","last_change":"2024-01-25T16:05:46.790680+0000","last_active":"2024-01-25T16:06:49.271125+0000","last_peered":"2024-01-25T16:06:49.271125+0000","last_clean":"2024-01-25T16:06:49.271125+0000","last_became_active":"2024-01-25T16:05:46.790330+0000","last_became_peered":"2024-01-25T16:05:46.790330+0000","last_unstale":"2024-01-25T16:06:49.271125+0000","last_undegraded":"2024-01-25T16:06:49.271125+0000","last_fullsized":"2024-01-25T16:06:49.271125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:33.452484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271871+0000","last_change":"2024-01-25T16:05:46.802958+0000","last_active":"2024-01-25T16:06:49.271871+0000","last_peered":"2024-01-25T16:06:49.271871+0000","last_clean":"2024-01-25T16:06:49.271871+0000","last_became_active":"2024-01-25T16:05:46.802606+0000","last_became_peered":"2024-01-25T16:05:46.802606+0000","last_unstale":"2024-01-25T16:06:49.271871+0000","last_undegraded":"2024-01-25T16:06:49.271871+0000","last_fullsized":"2024-01-25T16:06:49.271871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:57:44.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556353+0000","last_change":"2024-01-25T16:05:46.794974+0000","last_active":"2024-01-25T16:06:49.556353+0000","last_peered":"2024-01-25T16:06:49.556353+0000","last_clean":"2024-01-25T16:06:49.556353+0000","last_became_active":"2024-01-25T16:05:46.794629+0000","last_became_peered":"2024-01-25T16:05:46.794629+0000","last_unstale":"2024-01-25T16:06:49.556353+0000","last_undegraded":"2024-01-25T16:06:49.556353+0000","last_fullsized":"2024-01-25T16:06:49.556353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:19:22.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271462+0000","last_change":"2024-01-25T16:05:46.798725+0000","last_active":"2024-01-25T16:06:49.271462+0000","last_peered":"2024-01-25T16:06:49.271462+0000","last_clean":"2024-01-25T16:06:49.271462+0000","last_became_active":"2024-01-25T16:05:46.798373+0000","last_became_peered":"2024-01-25T16:05:46.798373+0000","last_unstale":"2024-01-25T16:06:49.271462+0000","last_undegraded":"2024-01-25T16:06:49.271462+0000","last_fullsized":"2024-01-25T16:06:49.271462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:55:31.117988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866545+0000","last_change":"2024-01-25T16:05:46.795421+0000","last_active":"2024-01-25T16:06:49.866545+0000","last_peered":"2024-01-25T16:06:49.866545+0000","last_clean":"2024-01-25T16:06:49.866545+0000","last_became_active":"2024-01-25T16:05:46.795068+0000","last_became_peered":"2024-01-25T16:05:46.795068+0000","last_unstale":"2024-01-25T16:06:49.866545+0000","last_undegraded":"2024-01-25T16:06:49.866545+0000","last_fullsized":"2024-01-25T16:06:49.866545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:20:46.490531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143322+0000","last_change":"2024-01-25T16:05:46.802408+0000","last_active":"2024-01-25T16:06:50.143322+0000","last_peered":"2024-01-25T16:06:50.143322+0000","last_clean":"2024-01-25T16:06:50.143322+0000","last_became_active":"2024-01-25T16:05:46.802104+0000","last_became_peered":"2024-01-25T16:05:46.802104+0000","last_unstale":"2024-01-25T16:06:50.143322+0000","last_undegraded":"2024-01-25T16:06:50.143322+0000","last_fullsized":"2024-01-25T16:06:50.143322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:00:11.366282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.276111+0000","last_change":"2024-01-25T16:06:49.276111+0000","last_active":"2024-01-25T16:06:49.276111+0000","last_peered":"2024-01-25T16:06:49.276111+0000","last_clean":"2024-01-25T16:06:49.276111+0000","last_became_active":"2024-01-25T16:06:49.275665+0000","last_became_peered":"2024-01-25T16:06:49.275665+0000","last_unstale":"2024-01-25T16:06:49.276111+0000","last_undegraded":"2024-01-25T16:06:49.276111+0000","last_fullsized":"2024-01-25T16:06:49.276111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:09:56.382891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60631,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":455,"num_read_kb":291,"num_write":152,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":181893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.4710000000000001,"5min":0.71199999999999997,"15min":0.58599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":15.404999999999999,"5min":15.404999999999999,"15min":15.404999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.45900000000000002,"15min":0.44400000000000001},"min":{"1min":0.38100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.46400000000000002},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.54000000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.53900000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:05:56 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.879,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.748},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91500000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73199999999999998,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.68300000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94199999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84199999999999997}]}]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1108419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.438}]},{"osd":1,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.629}]},{"osd":5,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.81200000000000006,"5min":1.171,"15min":1.171},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93000000000000005,"5min":1.139,"15min":1.139},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.75900000000000001,"15min":0.76900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1639999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84499999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96999999999999997,"5min":1.272,"15min":1.272},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98399999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80900000000000005,"15min":0.82899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":1.331,"15min":1.331},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 25 16:06:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79300000000000004,"15min":0.78700000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.872}]}]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":460543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53400000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.498,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.375}]},{"osd":1,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.748,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63300000000000001,"5min":0.997,"15min":0.997},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.79700000000000004,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":17.481999999999999,"5min":17.481999999999999,"15min":17.481999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69299999999999995,"15min":0.71099999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.4219999999999999,"5min":0.93100000000000005,"15min":0.84899999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":16.093,"5min":16.093,"15min":16.093},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.03,"15min":1.03},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86099999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.167,"15min":1.167},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.84599999999999997,"15min":0.872},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91200000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.67300000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":0.93000000000000005,"15min":0.84699999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":16.106000000000002,"5min":16.106000000000002,"15min":16.106000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.80700000000000005,"15min":0.82899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89600000000000002,"5min":1.23,"15min":1.23},"last":0.63300000000000001}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72699999999999998,"15min":0.73899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.80900000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":0.94299999999999995,"15min":0.83399999999999996},"min":{"1min":0.61599999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":17.495000000000001,"5min":17.495000000000001,"15min":17.495000000000001},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":460463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.878}]},{"osd":2,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.46500000000000002,"15min":0.45400000000000001},"min":{"1min":0.33700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.747},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.78800000000000003}]},{"osd":6,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.86699999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.92100000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.98799999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1572864,"data_stored":1052392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.44800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.28299999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.434,"15min":0.40600000000000003},"min":{"1min":0.313,"5min":0.247,"15min":0.247},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3610000000000002,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.61699999999999999,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.66700000000000004}]},{"osd":6,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.70899999999999996,"15min":0.65100000000000002},"min":{"1min":0.55500000000000005,"5min":0.443,"15min":0.443},"max":{"1min":5.931,"5min":5.931,"15min":5.931},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.628},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.73899999999999999,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.53800000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.1269999999999998,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.72199999999999998},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 25 16:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81399999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42299999999999999,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60099999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45200000000000001,"15min":0.42899999999999999},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":0.77300000000000002,"15min":0.66900000000000004},"min":{"1min":0.47699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":12.151,"5min":12.151,"15min":12.151},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65200000000000002,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":1,"15min":1},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.48599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":1.131,"15min":1.131},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":0.88300000000000001,"15min":0.71299999999999997},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":19.369,"5min":19.369,"15min":19.369},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83099999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.40500000000000003}]},{"osd":7,"last update":"Thu Jan 25 16:06:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.51500000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48799999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.931,"5min":1.0109999999999999,"15min":0.85499999999999998},"min":{"1min":0.58599999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":19.265999999999998,"5min":19.265999999999998,"15min":19.265999999999998},"last":0.71499999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.0429999999999999,"15min":0.90100000000000002},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":19.245000000000001,"5min":19.245000000000001,"15min":19.245000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.893,"5min":1.085,"15min":0.95099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":19.334,"5min":19.334,"15min":19.334},"last":0.79600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":516356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.873,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.378}]},{"osd":3,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61499999999999999,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.747}]},{"osd":4,"last update":"Thu Jan 25 16:06:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.34499999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.871,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81899999999999995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63400000000000001,"15min":0.58799999999999997},"min":{"1min":0.55300000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":1.109,"15min":1.109},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.52300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.002},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96499999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.63700000000000001,"15min":0.59999999999999998},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0680000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.97199999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.436},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.71699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.048,"5min":1.155,"15min":1.155},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.74399999999999999,"15min":0.70999999999999996},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.626}]},{"osd":9,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.55500000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.046,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77300000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80000000000000004,"15min":0.79400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.089,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.016,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59599999999999997}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":461203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 25 16:06:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.44400000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68400000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.77700000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.48899999999999999}]},{"osd":3,"last update":"Thu Jan 25 16:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35599999999999998}]},{"osd":4,"last update":"Thu Jan 25 16:06:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75,"5min":1.012,"15min":1.012},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:06:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 25 16:05:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.307},{"interface":"front","average":{"1min":0.501,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.40500000000000003}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.45200000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.746,"15min":0.77500000000000002},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78700000000000003,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.77200000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.505}]},{"osd":9,"last update":"Thu Jan 25 16:06:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.71699999999999997,"15min":0.74099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88900000000000001,"5min":1.115,"15min":1.115},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.71599999999999997,"15min":0.73699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92400000000000004,"5min":1.036,"15min":1.036},"last":0.47399999999999998}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.79200000000000004,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.81799999999999995,"5min":1.107,"15min":1.107},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.77600000000000002,"15min":0.79400000000000004},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90300000000000002,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.52600000000000002}]},{"osd":11,"last update":"Thu Jan 25 16:05:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.54600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":518062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 25 16:06:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83999999999999997,"5min":0.873,"15min":0.873},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57899999999999996,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.871,"15min":0.871},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.3360000000000001,"5min":0.74299999999999999,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":16.850999999999999,"5min":16.850999999999999,"15min":16.850999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 25 16:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.45000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32700000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:06:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.629},"min":{"1min":0.56499999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.462,"5min":0.80400000000000005,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":16.923999999999999,"5min":16.923999999999999,"15min":16.923999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499}]},{"osd":8,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.46300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.68100000000000005}]},{"osd":9,"last update":"Thu Jan 25 16:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:06:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.56699999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64300000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":460243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.32600000000000001},"min":{"1min":0.251,"5min":0.193,"15min":0.193},"max":{"1min":0.65700000000000003,"5min":0.755,"15min":0.755},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.36599999999999999,"15min":0.34200000000000003},"min":{"1min":0.25800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.65000000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.432}]},{"osd":1,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69299999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76400000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.877},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79900000000000004,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88100000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82499999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84599999999999997,"5min":1.014,"15min":1.014},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84799999999999998,"5min":1,"15min":1},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.87,"15min":0.87},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 25 16:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81599999999999995,"5min":0.879,"15min":0.879},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63400000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.68100000000000005}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.52500000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93799999999999994,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.749}]},{"osd":11,"last update":"Thu Jan 25 16:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.85799999999999998}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.438,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.376,"15min":0.35399999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86099999999999999,"5min":0.877,"15min":0.877},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.47899999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78600000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:06:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 25 16:06:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.878},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 25 16:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.86299999999999999}]},{"osd":8,"last update":"Thu Jan 25 16:06:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89200000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95899999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.84599999999999997}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91400000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95799999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.72899999999999998}]},{"osd":11,"last update":"Thu Jan 25 16:06:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79600000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1585152,"data_stored":1052076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.751,"15min":0.751},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59099999999999997,"15min":0.56100000000000005},"min":{"1min":0.52400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.436,"15min":0.42099999999999999},"min":{"1min":0.33800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.317}]},{"osd":3,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.443}]},{"osd":4,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.58599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.54800000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 25 16:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 25 16:06:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89500000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.54100000000000004,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47099999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54300000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.61199999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.63600000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-25T16:07:03.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph pg dump --format=json 2024-01-25T16:07:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:03 smithi152 ceph-mon[36015]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:03 smithi076 ceph-mon[35642]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 2 B/s, 1 keys/s, 0 objects/s recovering 2024-01-25T16:07:04.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:04 smithi033 ceph-mon[31417]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:04.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:04 smithi152 ceph-mon[36015]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:04.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:04 smithi076 ceph-mon[35642]: from='client.14970 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 -- 172.21.15.33:0/17909913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa8106680 msgr2=0x7feaa8106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 --2- 172.21.15.33:0/17909913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa8106680 0x7feaa8106a60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fea98009990 tx=0x7fea98009ca0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 -- 172.21.15.33:0/17909913 shutdown_connections 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 --2- 172.21.15.33:0/17909913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa81010d0 0x7feaa810d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 --2- 172.21.15.33:0/17909913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa8106680 0x7feaa8106a60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 --2- 172.21.15.33:0/17909913 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 0x7feaa8100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 -- 172.21.15.33:0/17909913 >> 172.21.15.33:0/17909913 conn(0x7feaa8076730 msgr2=0x7feaa8076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:05.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 -- 172.21.15.33:0/17909913 shutdown_connections 2024-01-25T16:07:05.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.272+0000 7feaad676700 1 -- 172.21.15.33:0/17909913 wait complete. 2024-01-25T16:07:05.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 Processor -- start 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 -- start start 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 0x7feaa8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa8106680 0x7feaa81a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81998a0 con 0x7feaa81010d0 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81a4080 con 0x7feaa8106680 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaad676700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81a41c0 con 0x7feaa8100710 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7fea9ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53946/0 (socket says 172.21.15.33:53946) 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7fea9ffff700 1 -- 172.21.15.33:0/2156879979 learned_addr learned my addr 172.21.15.33:0/2156879979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.273+0000 7feaa6ffd700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 0x7feaa8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaa77fe700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa8106680 0x7feaa81a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:05.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 -- 172.21.15.33:0/2156879979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 msgr2=0x7feaa8198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 0x7feaa8198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 -- 172.21.15.33:0/2156879979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa8106680 msgr2=0x7feaa81a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa8106680 0x7feaa81a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 -- 172.21.15.33:0/2156879979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980095f0 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7fea9ffff700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea9000d980 tx=0x7fea9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea900098d0 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa81a4470 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea90010460 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9000f620 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.274+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa81a49c0 con 0x7feaa81010d0 2024-01-25T16:07:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.275+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa8108c80 con 0x7feaa81010d0 2024-01-25T16:07:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.276+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fea90010860 con 0x7feaa81010d0 2024-01-25T16:07:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.277+0000 7feaa4ff9700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 0x7fea940a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.277+0000 7feaa6ffd700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 0x7fea940a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.277+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fea900cca90 con 0x7feaa81010d0 2024-01-25T16:07:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.277+0000 7feaa6ffd700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 0x7fea940a38c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea9800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:05.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.281+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fea90095d70 con 0x7feaa81010d0 2024-01-25T16:07:05.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:05 smithi033 ceph-mon[31417]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:05.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.479+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feaa8110660 con 0x7fea940a1400 2024-01-25T16:07:05.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.490+0000 7feaa4ff9700 1 -- 172.21.15.33:0/2156879979 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+344436 (secure 0 0 0) 0x7feaa8110660 con 0x7fea940a1400 2024-01-25T16:07:05.498 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:05.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 msgr2=0x7fea940a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:05.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 0x7fea940a38c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fea98006010 tx=0x7fea9800b540 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 msgr2=0x7feaa8199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fea9000d980 tx=0x7fea9000dc90 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 shutdown_connections 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fea940a1400 0x7fea940a38c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feaa8106680 0x7feaa81a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feaa8100710 0x7feaa8198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 --2- 172.21.15.33:0/2156879979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feaa81010d0 0x7feaa8199000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 >> 172.21.15.33:0/2156879979 conn(0x7feaa8076730 msgr2=0x7feaa80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 shutdown_connections 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:05.494+0000 7feaad676700 1 -- 172.21.15.33:0/2156879979 wait complete. 2024-01-25T16:07:05.501 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-01-25T16:07:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:05 smithi152 ceph-mon[36015]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:05 smithi076 ceph-mon[35642]: pgmap v326: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:06.095 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-01-25T16:07:04.118293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651233,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":591,"num_read_kb":410,"num_write":335,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1180770,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":366764,"kb_used_data":13392,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124575060,"statfs":{"total":1151940427776,"available":1151564861440,"internally_reserved":0,"allocated":13713408,"data_stored":7470399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003242"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268274+0000","last_change":"2024-01-25T16:05:46.796305+0000","last_active":"2024-01-25T16:06:49.268274+0000","last_peered":"2024-01-25T16:06:49.268274+0000","last_clean":"2024-01-25T16:06:49.268274+0000","last_became_active":"2024-01-25T16:05:46.795989+0000","last_became_peered":"2024-01-25T16:05:46.795989+0000","last_unstale":"2024-01-25T16:06:49.268274+0000","last_undegraded":"2024-01-25T16:06:49.268274+0000","last_fullsized":"2024-01-25T16:06:49.268274+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:45.764391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274441+0000","last_change":"2024-01-25T16:05:46.798335+0000","last_active":"2024-01-25T16:06:49.274441+0000","last_peered":"2024-01-25T16:06:49.274441+0000","last_clean":"2024-01-25T16:06:49.274441+0000","last_became_active":"2024-01-25T16:05:46.797743+0000","last_became_peered":"2024-01-25T16:05:46.797743+0000","last_unstale":"2024-01-25T16:06:49.274441+0000","last_undegraded":"2024-01-25T16:06:49.274441+0000","last_fullsized":"2024-01-25T16:06:49.274441+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:06:01.335919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274046+0000","last_change":"2024-01-25T16:05:46.803057+0000","last_active":"2024-01-25T16:06:49.274046+0000","last_peered":"2024-01-25T16:06:49.274046+0000","last_clean":"2024-01-25T16:06:49.274046+0000","last_became_active":"2024-01-25T16:05:46.802706+0000","last_became_peered":"2024-01-25T16:05:46.802706+0000","last_unstale":"2024-01-25T16:06:49.274046+0000","last_undegraded":"2024-01-25T16:06:49.274046+0000","last_fullsized":"2024-01-25T16:06:49.274046+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:27.159532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.691017+0000","last_change":"2024-01-25T16:05:46.796172+0000","last_active":"2024-01-25T16:06:49.691017+0000","last_peered":"2024-01-25T16:06:49.691017+0000","last_clean":"2024-01-25T16:06:49.691017+0000","last_became_active":"2024-01-25T16:05:46.795848+0000","last_became_peered":"2024-01-25T16:05:46.795848+0000","last_unstale":"2024-01-25T16:06:49.691017+0000","last_undegraded":"2024-01-25T16:06:49.691017+0000","last_fullsized":"2024-01-25T16:06:49.691017+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:41:07.553896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274776+0000","last_change":"2024-01-25T16:05:46.794790+0000","last_active":"2024-01-25T16:06:49.274776+0000","last_peered":"2024-01-25T16:06:49.274776+0000","last_clean":"2024-01-25T16:06:49.274776+0000","last_became_active":"2024-01-25T16:05:46.794464+0000","last_became_peered":"2024-01-25T16:05:46.794464+0000","last_unstale":"2024-01-25T16:06:49.274776+0000","last_undegraded":"2024-01-25T16:06:49.274776+0000","last_fullsized":"2024-01-25T16:06:49.274776+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:19:26.415959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.569416+0000","last_change":"2024-01-25T16:06:49.569416+0000","last_active":"2024-01-25T16:06:49.569416+0000","last_peered":"2024-01-25T16:06:49.569416+0000","last_clean":"2024-01-25T16:06:49.569416+0000","last_became_active":"2024-01-25T16:06:49.569006+0000","last_became_peered":"2024-01-25T16:06:49.569006+0000","last_unstale":"2024-01-25T16:06:49.569416+0000","last_undegraded":"2024-01-25T16:06:49.569416+0000","last_fullsized":"2024-01-25T16:06:49.569416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:38.469353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,7],"acting":[10,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568423+0000","last_change":"2024-01-25T16:05:46.801432+0000","last_active":"2024-01-25T16:06:49.568423+0000","last_peered":"2024-01-25T16:06:49.568423+0000","last_clean":"2024-01-25T16:06:49.568423+0000","last_became_active":"2024-01-25T16:05:46.796673+0000","last_became_peered":"2024-01-25T16:05:46.796673+0000","last_unstale":"2024-01-25T16:06:49.568423+0000","last_undegraded":"2024-01-25T16:06:49.568423+0000","last_fullsized":"2024-01-25T16:06:49.568423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T04:02:37.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270781+0000","last_change":"2024-01-25T16:05:46.805885+0000","last_active":"2024-01-25T16:06:49.270781+0000","last_peered":"2024-01-25T16:06:49.270781+0000","last_clean":"2024-01-25T16:06:49.270781+0000","last_became_active":"2024-01-25T16:05:46.805359+0000","last_became_peered":"2024-01-25T16:05:46.805359+0000","last_unstale":"2024-01-25T16:06:49.270781+0000","last_undegraded":"2024-01-25T16:06:49.270781+0000","last_fullsized":"2024-01-25T16:06:49.270781+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:44:54.641646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866917+0000","last_change":"2024-01-25T16:05:46.791944+0000","last_active":"2024-01-25T16:06:49.866917+0000","last_peered":"2024-01-25T16:06:49.866917+0000","last_clean":"2024-01-25T16:06:49.866917+0000","last_became_active":"2024-01-25T16:05:46.791637+0000","last_became_peered":"2024-01-25T16:05:46.791637+0000","last_unstale":"2024-01-25T16:06:49.866917+0000","last_undegraded":"2024-01-25T16:06:49.866917+0000","last_fullsized":"2024-01-25T16:06:49.866917+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:54:20.508990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268025+0000","last_change":"2024-01-25T16:05:46.790961+0000","last_active":"2024-01-25T16:06:49.268025+0000","last_peered":"2024-01-25T16:06:49.268025+0000","last_clean":"2024-01-25T16:06:49.268025+0000","last_became_active":"2024-01-25T16:05:46.790664+0000","last_became_peered":"2024-01-25T16:05:46.790664+0000","last_unstale":"2024-01-25T16:06:49.268025+0000","last_undegraded":"2024-01-25T16:06:49.268025+0000","last_fullsized":"2024-01-25T16:06:49.268025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:24:57.615304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568162+0000","last_change":"2024-01-25T16:05:46.795205+0000","last_active":"2024-01-25T16:06:49.568162+0000","last_peered":"2024-01-25T16:06:49.568162+0000","last_clean":"2024-01-25T16:06:49.568162+0000","last_became_active":"2024-01-25T16:05:46.794894+0000","last_became_peered":"2024-01-25T16:05:46.794894+0000","last_unstale":"2024-01-25T16:06:49.568162+0000","last_undegraded":"2024-01-25T16:06:49.568162+0000","last_fullsized":"2024-01-25T16:06:49.568162+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:57:37.069575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.690975+0000","last_change":"2024-01-25T16:05:46.795473+0000","last_active":"2024-01-25T16:06:49.690975+0000","last_peered":"2024-01-25T16:06:49.690975+0000","last_clean":"2024-01-25T16:06:49.690975+0000","last_became_active":"2024-01-25T16:05:46.795155+0000","last_became_peered":"2024-01-25T16:05:46.795155+0000","last_unstale":"2024-01-25T16:06:49.690975+0000","last_undegraded":"2024-01-25T16:06:49.690975+0000","last_fullsized":"2024-01-25T16:06:49.690975+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:42:25.273806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143317+0000","last_change":"2024-01-25T16:05:46.792550+0000","last_active":"2024-01-25T16:06:50.143317+0000","last_peered":"2024-01-25T16:06:50.143317+0000","last_clean":"2024-01-25T16:06:50.143317+0000","last_became_active":"2024-01-25T16:05:46.792147+0000","last_became_peered":"2024-01-25T16:05:46.792147+0000","last_unstale":"2024-01-25T16:06:50.143317+0000","last_undegraded":"2024-01-25T16:06:50.143317+0000","last_fullsized":"2024-01-25T16:06:50.143317+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:05:08.117739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273636+0000","last_change":"2024-01-25T16:05:46.808812+0000","last_active":"2024-01-25T16:06:49.273636+0000","last_peered":"2024-01-25T16:06:49.273636+0000","last_clean":"2024-01-25T16:06:49.273636+0000","last_became_active":"2024-01-25T16:05:46.808077+0000","last_became_peered":"2024-01-25T16:05:46.808077+0000","last_unstale":"2024-01-25T16:06:49.273636+0000","last_undegraded":"2024-01-25T16:06:49.273636+0000","last_fullsized":"2024-01-25T16:06:49.273636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:56:38.329223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273636+0000","last_change":"2024-01-25T16:05:46.796315+0000","last_active":"2024-01-25T16:06:49.273636+0000","last_peered":"2024-01-25T16:06:49.273636+0000","last_clean":"2024-01-25T16:06:49.273636+0000","last_became_active":"2024-01-25T16:05:46.795998+0000","last_became_peered":"2024-01-25T16:05:46.795998+0000","last_unstale":"2024-01-25T16:06:49.273636+0000","last_undegraded":"2024-01-25T16:06:49.273636+0000","last_fullsized":"2024-01-25T16:06:49.273636+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:44:39.566605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866656+0000","last_change":"2024-01-25T16:05:46.798086+0000","last_active":"2024-01-25T16:06:49.866656+0000","last_peered":"2024-01-25T16:06:49.866656+0000","last_clean":"2024-01-25T16:06:49.866656+0000","last_became_active":"2024-01-25T16:05:46.797424+0000","last_became_peered":"2024-01-25T16:05:46.797424+0000","last_unstale":"2024-01-25T16:06:49.866656+0000","last_undegraded":"2024-01-25T16:06:49.866656+0000","last_fullsized":"2024-01-25T16:06:49.866656+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:43:04.721044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268488+0000","last_change":"2024-01-25T16:05:46.799058+0000","last_active":"2024-01-25T16:06:49.268488+0000","last_peered":"2024-01-25T16:06:49.268488+0000","last_clean":"2024-01-25T16:06:49.268488+0000","last_became_active":"2024-01-25T16:05:46.798708+0000","last_became_peered":"2024-01-25T16:05:46.798708+0000","last_unstale":"2024-01-25T16:06:49.268488+0000","last_undegraded":"2024-01-25T16:06:49.268488+0000","last_fullsized":"2024-01-25T16:06:49.268488+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:04:48.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272560+0000","last_change":"2024-01-25T16:05:46.791726+0000","last_active":"2024-01-25T16:06:49.272560+0000","last_peered":"2024-01-25T16:06:49.272560+0000","last_clean":"2024-01-25T16:06:49.272560+0000","last_became_active":"2024-01-25T16:05:46.791301+0000","last_became_peered":"2024-01-25T16:05:46.791301+0000","last_unstale":"2024-01-25T16:06:49.272560+0000","last_undegraded":"2024-01-25T16:06:49.272560+0000","last_fullsized":"2024-01-25T16:06:49.272560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:43:25.973750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270722+0000","last_change":"2024-01-25T16:05:46.795523+0000","last_active":"2024-01-25T16:06:49.270722+0000","last_peered":"2024-01-25T16:06:49.270722+0000","last_clean":"2024-01-25T16:06:49.270722+0000","last_became_active":"2024-01-25T16:05:46.794965+0000","last_became_peered":"2024-01-25T16:05:46.794965+0000","last_unstale":"2024-01-25T16:06:49.270722+0000","last_undegraded":"2024-01-25T16:06:49.270722+0000","last_fullsized":"2024-01-25T16:06:49.270722+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:13:33.793210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272570+0000","last_change":"2024-01-25T16:05:45.764981+0000","last_active":"2024-01-25T16:06:49.272570+0000","last_peered":"2024-01-25T16:06:49.272570+0000","last_clean":"2024-01-25T16:06:49.272570+0000","last_became_active":"2024-01-25T16:05:45.764616+0000","last_became_peered":"2024-01-25T16:05:45.764616+0000","last_unstale":"2024-01-25T16:06:49.272570+0000","last_undegraded":"2024-01-25T16:06:49.272570+0000","last_fullsized":"2024-01-25T16:06:49.272570+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:26:19.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270713+0000","last_change":"2024-01-25T16:05:46.800786+0000","last_active":"2024-01-25T16:06:49.270713+0000","last_peered":"2024-01-25T16:06:49.270713+0000","last_clean":"2024-01-25T16:06:49.270713+0000","last_became_active":"2024-01-25T16:05:46.799666+0000","last_became_peered":"2024-01-25T16:05:46.799666+0000","last_unstale":"2024-01-25T16:06:49.270713+0000","last_undegraded":"2024-01-25T16:06:49.270713+0000","last_fullsized":"2024-01-25T16:06:49.270713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:49:03.942394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271859+0000","last_change":"2024-01-25T16:05:45.773627+0000","last_active":"2024-01-25T16:06:49.271859+0000","last_peered":"2024-01-25T16:06:49.271859+0000","last_clean":"2024-01-25T16:06:49.271859+0000","last_became_active":"2024-01-25T16:05:45.773383+0000","last_became_peered":"2024-01-25T16:05:45.773383+0000","last_unstale":"2024-01-25T16:06:49.271859+0000","last_undegraded":"2024-01-25T16:06:49.271859+0000","last_fullsized":"2024-01-25T16:06:49.271859+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:14:22.536001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274232+0000","last_change":"2024-01-25T16:05:46.801791+0000","last_active":"2024-01-25T16:06:49.274232+0000","last_peered":"2024-01-25T16:06:49.274232+0000","last_clean":"2024-01-25T16:06:49.274232+0000","last_became_active":"2024-01-25T16:05:46.801487+0000","last_became_peered":"2024-01-25T16:05:46.801487+0000","last_unstale":"2024-01-25T16:06:49.274232+0000","last_undegraded":"2024-01-25T16:06:49.274232+0000","last_fullsized":"2024-01-25T16:06:49.274232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:45:07.874907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275321+0000","last_change":"2024-01-25T16:05:45.774708+0000","last_active":"2024-01-25T16:06:49.275321+0000","last_peered":"2024-01-25T16:06:49.275321+0000","last_clean":"2024-01-25T16:06:49.275321+0000","last_became_active":"2024-01-25T16:05:45.774401+0000","last_became_peered":"2024-01-25T16:05:45.774401+0000","last_unstale":"2024-01-25T16:06:49.275321+0000","last_undegraded":"2024-01-25T16:06:49.275321+0000","last_fullsized":"2024-01-25T16:06:49.275321+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:01:46.786703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866442+0000","last_change":"2024-01-25T16:05:46.794386+0000","last_active":"2024-01-25T16:06:49.866442+0000","last_peered":"2024-01-25T16:06:49.866442+0000","last_clean":"2024-01-25T16:06:49.866442+0000","last_became_active":"2024-01-25T16:05:46.793909+0000","last_became_peered":"2024-01-25T16:05:46.793909+0000","last_unstale":"2024-01-25T16:06:49.866442+0000","last_undegraded":"2024-01-25T16:06:49.866442+0000","last_fullsized":"2024-01-25T16:06:49.866442+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:10:24.736904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275004+0000","last_change":"2024-01-25T16:05:45.767804+0000","last_active":"2024-01-25T16:06:49.275004+0000","last_peered":"2024-01-25T16:06:49.275004+0000","last_clean":"2024-01-25T16:06:49.275004+0000","last_became_active":"2024-01-25T16:05:45.767491+0000","last_became_peered":"2024-01-25T16:05:45.767491+0000","last_unstale":"2024-01-25T16:06:49.275004+0000","last_undegraded":"2024-01-25T16:06:49.275004+0000","last_fullsized":"2024-01-25T16:06:49.275004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:20:31.331950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556631+0000","last_change":"2024-01-25T16:05:46.803061+0000","last_active":"2024-01-25T16:06:49.556631+0000","last_peered":"2024-01-25T16:06:49.556631+0000","last_clean":"2024-01-25T16:06:49.556631+0000","last_became_active":"2024-01-25T16:05:46.802756+0000","last_became_peered":"2024-01-25T16:05:46.802756+0000","last_unstale":"2024-01-25T16:06:49.556631+0000","last_undegraded":"2024-01-25T16:06:49.556631+0000","last_fullsized":"2024-01-25T16:06:49.556631+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:31:53.107217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.159128+0000","last_change":"2024-01-25T16:05:45.764304+0000","last_active":"2024-01-25T16:07:00.159128+0000","last_peered":"2024-01-25T16:07:00.159128+0000","last_clean":"2024-01-25T16:07:00.159128+0000","last_became_active":"2024-01-25T16:05:45.763977+0000","last_became_peered":"2024-01-25T16:05:45.763977+0000","last_unstale":"2024-01-25T16:07:00.159128+0000","last_undegraded":"2024-01-25T16:07:00.159128+0000","last_fullsized":"2024-01-25T16:07:00.159128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:16:07.031015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1012,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274510+0000","last_change":"2024-01-25T16:05:46.802631+0000","last_active":"2024-01-25T16:06:49.274510+0000","last_peered":"2024-01-25T16:06:49.274510+0000","last_clean":"2024-01-25T16:06:49.274510+0000","last_became_active":"2024-01-25T16:05:46.802305+0000","last_became_peered":"2024-01-25T16:05:46.802305+0000","last_unstale":"2024-01-25T16:06:49.274510+0000","last_undegraded":"2024-01-25T16:06:49.274510+0000","last_fullsized":"2024-01-25T16:06:49.274510+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:26:59.388515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267678+0000","last_change":"2024-01-25T16:05:45.762541+0000","last_active":"2024-01-25T16:06:49.267678+0000","last_peered":"2024-01-25T16:06:49.267678+0000","last_clean":"2024-01-25T16:06:49.267678+0000","last_became_active":"2024-01-25T16:05:45.762238+0000","last_became_peered":"2024-01-25T16:05:45.762238+0000","last_unstale":"2024-01-25T16:06:49.267678+0000","last_undegraded":"2024-01-25T16:06:49.267678+0000","last_fullsized":"2024-01-25T16:06:49.267678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:56:41.311268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556500+0000","last_change":"2024-01-25T16:05:46.794741+0000","last_active":"2024-01-25T16:06:49.556500+0000","last_peered":"2024-01-25T16:06:49.556500+0000","last_clean":"2024-01-25T16:06:49.556500+0000","last_became_active":"2024-01-25T16:05:46.794294+0000","last_became_peered":"2024-01-25T16:05:46.794294+0000","last_unstale":"2024-01-25T16:06:49.556500+0000","last_undegraded":"2024-01-25T16:06:49.556500+0000","last_fullsized":"2024-01-25T16:06:49.556500+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:43:00.876478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.142995+0000","last_change":"2024-01-25T16:05:45.766535+0000","last_active":"2024-01-25T16:06:50.142995+0000","last_peered":"2024-01-25T16:06:50.142995+0000","last_clean":"2024-01-25T16:06:50.142995+0000","last_became_active":"2024-01-25T16:05:45.765634+0000","last_became_peered":"2024-01-25T16:05:45.765634+0000","last_unstale":"2024-01-25T16:06:50.142995+0000","last_undegraded":"2024-01-25T16:06:50.142995+0000","last_fullsized":"2024-01-25T16:06:50.142995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:43:31.723438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270904+0000","last_change":"2024-01-25T16:05:46.793549+0000","last_active":"2024-01-25T16:06:49.270904+0000","last_peered":"2024-01-25T16:06:49.270904+0000","last_clean":"2024-01-25T16:06:49.270904+0000","last_became_active":"2024-01-25T16:05:46.793217+0000","last_became_peered":"2024-01-25T16:05:46.793217+0000","last_unstale":"2024-01-25T16:06:49.270904+0000","last_undegraded":"2024-01-25T16:06:49.270904+0000","last_fullsized":"2024-01-25T16:06:49.270904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:17:34.531729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556305+0000","last_change":"2024-01-25T16:05:45.768755+0000","last_active":"2024-01-25T16:06:49.556305+0000","last_peered":"2024-01-25T16:06:49.556305+0000","last_clean":"2024-01-25T16:06:49.556305+0000","last_became_active":"2024-01-25T16:05:45.768457+0000","last_became_peered":"2024-01-25T16:05:45.768457+0000","last_unstale":"2024-01-25T16:06:49.556305+0000","last_undegraded":"2024-01-25T16:06:49.556305+0000","last_fullsized":"2024-01-25T16:06:49.556305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:37:46.116204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270589+0000","last_change":"2024-01-25T16:05:46.792590+0000","last_active":"2024-01-25T16:06:49.270589+0000","last_peered":"2024-01-25T16:06:49.270589+0000","last_clean":"2024-01-25T16:06:49.270589+0000","last_became_active":"2024-01-25T16:05:46.792246+0000","last_became_peered":"2024-01-25T16:05:46.792246+0000","last_unstale":"2024-01-25T16:06:49.270589+0000","last_undegraded":"2024-01-25T16:06:49.270589+0000","last_fullsized":"2024-01-25T16:06:49.270589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:11:31.717248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:59.157176+0000","last_change":"2024-01-25T16:05:45.765720+0000","last_active":"2024-01-25T16:06:59.157176+0000","last_peered":"2024-01-25T16:06:59.157176+0000","last_clean":"2024-01-25T16:06:59.157176+0000","last_became_active":"2024-01-25T16:05:45.765336+0000","last_became_peered":"2024-01-25T16:05:45.765336+0000","last_unstale":"2024-01-25T16:06:59.157176+0000","last_undegraded":"2024-01-25T16:06:59.157176+0000","last_fullsized":"2024-01-25T16:06:59.157176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:26.016352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866840+0000","last_change":"2024-01-25T16:05:46.795603+0000","last_active":"2024-01-25T16:06:49.866840+0000","last_peered":"2024-01-25T16:06:49.866840+0000","last_clean":"2024-01-25T16:06:49.866840+0000","last_became_active":"2024-01-25T16:05:46.794725+0000","last_became_peered":"2024-01-25T16:05:46.794725+0000","last_unstale":"2024-01-25T16:06:49.866840+0000","last_undegraded":"2024-01-25T16:06:49.866840+0000","last_fullsized":"2024-01-25T16:06:49.866840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:08:28.823911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568169+0000","last_change":"2024-01-25T16:05:45.769249+0000","last_active":"2024-01-25T16:06:49.568169+0000","last_peered":"2024-01-25T16:06:49.568169+0000","last_clean":"2024-01-25T16:06:49.568169+0000","last_became_active":"2024-01-25T16:05:45.768939+0000","last_became_peered":"2024-01-25T16:05:45.768939+0000","last_unstale":"2024-01-25T16:06:49.568169+0000","last_undegraded":"2024-01-25T16:06:49.568169+0000","last_fullsized":"2024-01-25T16:06:49.568169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:32:46.727035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.557905+0000","last_change":"2024-01-25T16:06:49.557905+0000","last_active":"2024-01-25T16:06:49.557905+0000","last_peered":"2024-01-25T16:06:49.557905+0000","last_clean":"2024-01-25T16:06:49.557905+0000","last_became_active":"2024-01-25T16:06:49.557559+0000","last_became_peered":"2024-01-25T16:06:49.557559+0000","last_unstale":"2024-01-25T16:06:49.557905+0000","last_undegraded":"2024-01-25T16:06:49.557905+0000","last_fullsized":"2024-01-25T16:06:49.557905+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:25:50.375970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556885+0000","last_change":"2024-01-25T16:05:45.758681+0000","last_active":"2024-01-25T16:06:49.556885+0000","last_peered":"2024-01-25T16:06:49.556885+0000","last_clean":"2024-01-25T16:06:49.556885+0000","last_became_active":"2024-01-25T16:05:45.758394+0000","last_became_peered":"2024-01-25T16:05:45.758394+0000","last_unstale":"2024-01-25T16:06:49.556885+0000","last_undegraded":"2024-01-25T16:06:49.556885+0000","last_fullsized":"2024-01-25T16:06:49.556885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:55:50.498848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866562+0000","last_change":"2024-01-25T16:05:46.794286+0000","last_active":"2024-01-25T16:06:49.866562+0000","last_peered":"2024-01-25T16:06:49.866562+0000","last_clean":"2024-01-25T16:06:49.866562+0000","last_became_active":"2024-01-25T16:05:46.793949+0000","last_became_peered":"2024-01-25T16:05:46.793949+0000","last_unstale":"2024-01-25T16:06:49.866562+0000","last_undegraded":"2024-01-25T16:06:49.866562+0000","last_fullsized":"2024-01-25T16:06:49.866562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:47:02.196054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":194,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.158530+0000","last_change":"2024-01-25T16:05:45.761374+0000","last_active":"2024-01-25T16:07:00.158530+0000","last_peered":"2024-01-25T16:07:00.158530+0000","last_clean":"2024-01-25T16:07:00.158530+0000","last_became_active":"2024-01-25T16:05:45.761060+0000","last_became_peered":"2024-01-25T16:05:45.761060+0000","last_unstale":"2024-01-25T16:07:00.158530+0000","last_undegraded":"2024-01-25T16:07:00.158530+0000","last_fullsized":"2024-01-25T16:07:00.158530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:56:40.591127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56631,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":149,"num_read_kb":129,"num_write":27,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274945+0000","last_change":"2024-01-25T16:05:46.799163+0000","last_active":"2024-01-25T16:06:49.274945+0000","last_peered":"2024-01-25T16:06:49.274945+0000","last_clean":"2024-01-25T16:06:49.274945+0000","last_became_active":"2024-01-25T16:05:46.798771+0000","last_became_peered":"2024-01-25T16:05:46.798771+0000","last_unstale":"2024-01-25T16:06:49.274945+0000","last_undegraded":"2024-01-25T16:06:49.274945+0000","last_fullsized":"2024-01-25T16:06:49.274945+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:30:23.693300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.282719+0000","last_change":"2024-01-25T16:06:49.282204+0000","last_active":"2024-01-25T16:06:49.282719+0000","last_peered":"2024-01-25T16:06:49.282719+0000","last_clean":"2024-01-25T16:06:49.282719+0000","last_became_active":"2024-01-25T16:06:49.273746+0000","last_became_peered":"2024-01-25T16:06:49.273746+0000","last_unstale":"2024-01-25T16:06:49.282719+0000","last_undegraded":"2024-01-25T16:06:49.282719+0000","last_fullsized":"2024-01-25T16:06:49.282719+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:49.400315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143250+0000","last_change":"2024-01-25T16:05:46.792576+0000","last_active":"2024-01-25T16:06:50.143250+0000","last_peered":"2024-01-25T16:06:50.143250+0000","last_clean":"2024-01-25T16:06:50.143250+0000","last_became_active":"2024-01-25T16:05:46.792220+0000","last_became_peered":"2024-01-25T16:05:46.792220+0000","last_unstale":"2024-01-25T16:06:50.143250+0000","last_undegraded":"2024-01-25T16:06:50.143250+0000","last_fullsized":"2024-01-25T16:06:50.143250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:49:31.388289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556417+0000","last_change":"2024-01-25T16:05:45.767265+0000","last_active":"2024-01-25T16:06:49.556417+0000","last_peered":"2024-01-25T16:06:49.556417+0000","last_clean":"2024-01-25T16:06:49.556417+0000","last_became_active":"2024-01-25T16:05:45.766877+0000","last_became_peered":"2024-01-25T16:05:45.766877+0000","last_unstale":"2024-01-25T16:06:49.556417+0000","last_undegraded":"2024-01-25T16:06:49.556417+0000","last_fullsized":"2024-01-25T16:06:49.556417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:47:54.705160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568171+0000","last_change":"2024-01-25T16:05:46.801811+0000","last_active":"2024-01-25T16:06:49.568171+0000","last_peered":"2024-01-25T16:06:49.568171+0000","last_clean":"2024-01-25T16:06:49.568171+0000","last_became_active":"2024-01-25T16:05:46.799546+0000","last_became_peered":"2024-01-25T16:05:46.799546+0000","last_unstale":"2024-01-25T16:06:49.568171+0000","last_undegraded":"2024-01-25T16:06:49.568171+0000","last_fullsized":"2024-01-25T16:06:49.568171+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:18:52.900456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.158290+0000","last_change":"2024-01-25T16:05:45.769514+0000","last_active":"2024-01-25T16:07:00.158290+0000","last_peered":"2024-01-25T16:07:00.158290+0000","last_clean":"2024-01-25T16:07:00.158290+0000","last_became_active":"2024-01-25T16:05:45.768497+0000","last_became_peered":"2024-01-25T16:05:45.768497+0000","last_unstale":"2024-01-25T16:07:00.158290+0000","last_undegraded":"2024-01-25T16:07:00.158290+0000","last_fullsized":"2024-01-25T16:07:00.158290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:51:37.418948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567772+0000","last_change":"2024-01-25T16:05:46.798502+0000","last_active":"2024-01-25T16:06:49.567772+0000","last_peered":"2024-01-25T16:06:49.567772+0000","last_clean":"2024-01-25T16:06:49.567772+0000","last_became_active":"2024-01-25T16:05:46.798203+0000","last_became_peered":"2024-01-25T16:05:46.798203+0000","last_unstale":"2024-01-25T16:06:49.567772+0000","last_undegraded":"2024-01-25T16:06:49.567772+0000","last_fullsized":"2024-01-25T16:06:49.567772+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:31:53.098065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274509+0000","last_change":"2024-01-25T16:05:45.764827+0000","last_active":"2024-01-25T16:06:49.274509+0000","last_peered":"2024-01-25T16:06:49.274509+0000","last_clean":"2024-01-25T16:06:49.274509+0000","last_became_active":"2024-01-25T16:05:45.764466+0000","last_became_peered":"2024-01-25T16:05:45.764466+0000","last_unstale":"2024-01-25T16:06:49.274509+0000","last_undegraded":"2024-01-25T16:06:49.274509+0000","last_fullsized":"2024-01-25T16:06:49.274509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:57:17.252922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556199+0000","last_change":"2024-01-25T16:05:46.803960+0000","last_active":"2024-01-25T16:06:49.556199+0000","last_peered":"2024-01-25T16:06:49.556199+0000","last_clean":"2024-01-25T16:06:49.556199+0000","last_became_active":"2024-01-25T16:05:46.803246+0000","last_became_peered":"2024-01-25T16:05:46.803246+0000","last_unstale":"2024-01-25T16:06:49.556199+0000","last_undegraded":"2024-01-25T16:06:49.556199+0000","last_fullsized":"2024-01-25T16:06:49.556199+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:52:42.354813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275598+0000","last_change":"2024-01-25T16:05:45.765952+0000","last_active":"2024-01-25T16:06:49.275598+0000","last_peered":"2024-01-25T16:06:49.275598+0000","last_clean":"2024-01-25T16:06:49.275598+0000","last_became_active":"2024-01-25T16:05:45.765575+0000","last_became_peered":"2024-01-25T16:05:45.765575+0000","last_unstale":"2024-01-25T16:06:49.275598+0000","last_undegraded":"2024-01-25T16:06:49.275598+0000","last_fullsized":"2024-01-25T16:06:49.275598+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:12:07.423245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.558287+0000","last_change":"2024-01-25T16:06:49.558287+0000","last_active":"2024-01-25T16:06:49.558287+0000","last_peered":"2024-01-25T16:06:49.558287+0000","last_clean":"2024-01-25T16:06:49.558287+0000","last_became_active":"2024-01-25T16:06:49.557763+0000","last_became_peered":"2024-01-25T16:06:49.557763+0000","last_unstale":"2024-01-25T16:06:49.558287+0000","last_undegraded":"2024-01-25T16:06:49.558287+0000","last_fullsized":"2024-01-25T16:06:49.558287+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:28.662830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,8],"acting":[11,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271711+0000","last_change":"2024-01-25T16:05:45.765934+0000","last_active":"2024-01-25T16:06:49.271711+0000","last_peered":"2024-01-25T16:06:49.271711+0000","last_clean":"2024-01-25T16:06:49.271711+0000","last_became_active":"2024-01-25T16:05:45.765583+0000","last_became_peered":"2024-01-25T16:05:45.765583+0000","last_unstale":"2024-01-25T16:06:49.271711+0000","last_undegraded":"2024-01-25T16:06:49.271711+0000","last_fullsized":"2024-01-25T16:06:49.271711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:42:45.814218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267932+0000","last_change":"2024-01-25T16:05:46.794622+0000","last_active":"2024-01-25T16:06:49.267932+0000","last_peered":"2024-01-25T16:06:49.267932+0000","last_clean":"2024-01-25T16:06:49.267932+0000","last_became_active":"2024-01-25T16:05:46.794304+0000","last_became_peered":"2024-01-25T16:05:46.794304+0000","last_unstale":"2024-01-25T16:06:49.267932+0000","last_undegraded":"2024-01-25T16:06:49.267932+0000","last_fullsized":"2024-01-25T16:06:49.267932+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:48:31.605501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568387+0000","last_change":"2024-01-25T16:05:45.772412+0000","last_active":"2024-01-25T16:06:49.568387+0000","last_peered":"2024-01-25T16:06:49.568387+0000","last_clean":"2024-01-25T16:06:49.568387+0000","last_became_active":"2024-01-25T16:05:45.772074+0000","last_became_peered":"2024-01-25T16:05:45.772074+0000","last_unstale":"2024-01-25T16:06:49.568387+0000","last_undegraded":"2024-01-25T16:06:49.568387+0000","last_fullsized":"2024-01-25T16:06:49.568387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:42:46.858459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267946+0000","last_change":"2024-01-25T16:05:46.800980+0000","last_active":"2024-01-25T16:06:49.267946+0000","last_peered":"2024-01-25T16:06:49.267946+0000","last_clean":"2024-01-25T16:06:49.267946+0000","last_became_active":"2024-01-25T16:05:46.800689+0000","last_became_peered":"2024-01-25T16:05:46.800689+0000","last_unstale":"2024-01-25T16:06:49.267946+0000","last_undegraded":"2024-01-25T16:06:49.267946+0000","last_fullsized":"2024-01-25T16:06:49.267946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:15:05.197587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556613+0000","last_change":"2024-01-25T16:05:46.802971+0000","last_active":"2024-01-25T16:06:49.556613+0000","last_peered":"2024-01-25T16:06:49.556613+0000","last_clean":"2024-01-25T16:06:49.556613+0000","last_became_active":"2024-01-25T16:05:46.802669+0000","last_became_peered":"2024-01-25T16:05:46.802669+0000","last_unstale":"2024-01-25T16:06:49.556613+0000","last_undegraded":"2024-01-25T16:06:49.556613+0000","last_fullsized":"2024-01-25T16:06:49.556613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:23:01.261255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:07:00.157687+0000","last_change":"2024-01-25T16:05:45.763052+0000","last_active":"2024-01-25T16:07:00.157687+0000","last_peered":"2024-01-25T16:07:00.157687+0000","last_clean":"2024-01-25T16:07:00.157687+0000","last_became_active":"2024-01-25T16:05:45.762745+0000","last_became_peered":"2024-01-25T16:05:45.762745+0000","last_unstale":"2024-01-25T16:07:00.157687+0000","last_undegraded":"2024-01-25T16:07:00.157687+0000","last_fullsized":"2024-01-25T16:07:00.157687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:11:04.783033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271971+0000","last_change":"2024-01-25T16:05:46.799092+0000","last_active":"2024-01-25T16:06:49.271971+0000","last_peered":"2024-01-25T16:06:49.271971+0000","last_clean":"2024-01-25T16:06:49.271971+0000","last_became_active":"2024-01-25T16:05:46.798693+0000","last_became_peered":"2024-01-25T16:05:46.798693+0000","last_unstale":"2024-01-25T16:06:49.271971+0000","last_undegraded":"2024-01-25T16:06:49.271971+0000","last_fullsized":"2024-01-25T16:06:49.271971+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:26:29.504645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.690896+0000","last_change":"2024-01-25T16:05:45.760805+0000","last_active":"2024-01-25T16:06:49.690896+0000","last_peered":"2024-01-25T16:06:49.690896+0000","last_clean":"2024-01-25T16:06:49.690896+0000","last_became_active":"2024-01-25T16:05:45.760420+0000","last_became_peered":"2024-01-25T16:05:45.760420+0000","last_unstale":"2024-01-25T16:06:49.690896+0000","last_undegraded":"2024-01-25T16:06:49.690896+0000","last_fullsized":"2024-01-25T16:06:49.690896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:14:16.150271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":353,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567893+0000","last_change":"2024-01-25T16:04:34.048543+0000","last_active":"2024-01-25T16:06:49.567893+0000","last_peered":"2024-01-25T16:06:49.567893+0000","last_clean":"2024-01-25T16:06:49.567893+0000","last_became_active":"2024-01-25T16:04:34.048051+0000","last_became_peered":"2024-01-25T16:04:34.048051+0000","last_unstale":"2024-01-25T16:06:49.567893+0000","last_undegraded":"2024-01-25T16:06:49.567893+0000","last_fullsized":"2024-01-25T16:06:49.567893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:01:16.962963+0000","last_clean_scrub_stamp":"2024-01-25T16:01:16.962963+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:34:03.052940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143304+0000","last_change":"2024-01-25T16:05:46.793372+0000","last_active":"2024-01-25T16:06:50.143304+0000","last_peered":"2024-01-25T16:06:50.143304+0000","last_clean":"2024-01-25T16:06:50.143304+0000","last_became_active":"2024-01-25T16:05:46.792312+0000","last_became_peered":"2024-01-25T16:05:46.792312+0000","last_unstale":"2024-01-25T16:06:50.143304+0000","last_undegraded":"2024-01-25T16:06:50.143304+0000","last_fullsized":"2024-01-25T16:06:50.143304+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:34:12.261247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.567957+0000","last_change":"2024-01-25T16:05:45.776106+0000","last_active":"2024-01-25T16:06:49.567957+0000","last_peered":"2024-01-25T16:06:49.567957+0000","last_clean":"2024-01-25T16:06:49.567957+0000","last_became_active":"2024-01-25T16:05:45.775880+0000","last_became_peered":"2024-01-25T16:05:45.775880+0000","last_unstale":"2024-01-25T16:06:49.567957+0000","last_undegraded":"2024-01-25T16:06:49.567957+0000","last_fullsized":"2024-01-25T16:06:49.567957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:30:00.130932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.275103+0000","last_change":"2024-01-25T16:05:45.775465+0000","last_active":"2024-01-25T16:06:49.275103+0000","last_peered":"2024-01-25T16:06:49.275103+0000","last_clean":"2024-01-25T16:06:49.275103+0000","last_became_active":"2024-01-25T16:05:45.775068+0000","last_became_peered":"2024-01-25T16:05:45.775068+0000","last_unstale":"2024-01-25T16:06:49.275103+0000","last_undegraded":"2024-01-25T16:06:49.275103+0000","last_fullsized":"2024-01-25T16:06:49.275103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:29:27.858166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143263+0000","last_change":"2024-01-25T16:05:46.788033+0000","last_active":"2024-01-25T16:06:50.143263+0000","last_peered":"2024-01-25T16:06:50.143263+0000","last_clean":"2024-01-25T16:06:50.143263+0000","last_became_active":"2024-01-25T16:05:46.787700+0000","last_became_peered":"2024-01-25T16:05:46.787700+0000","last_unstale":"2024-01-25T16:06:50.143263+0000","last_undegraded":"2024-01-25T16:06:50.143263+0000","last_fullsized":"2024-01-25T16:06:50.143263+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:41:02.923376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.286189+0000","last_change":"2024-01-25T16:06:49.286189+0000","last_active":"2024-01-25T16:06:49.286189+0000","last_peered":"2024-01-25T16:06:49.286189+0000","last_clean":"2024-01-25T16:06:49.286189+0000","last_became_active":"2024-01-25T16:06:49.274442+0000","last_became_peered":"2024-01-25T16:06:49.274442+0000","last_unstale":"2024-01-25T16:06:49.286189+0000","last_undegraded":"2024-01-25T16:06:49.286189+0000","last_fullsized":"2024-01-25T16:06:49.286189+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:34:14.971542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272127+0000","last_change":"2024-01-25T16:05:46.810382+0000","last_active":"2024-01-25T16:06:49.272127+0000","last_peered":"2024-01-25T16:06:49.272127+0000","last_clean":"2024-01-25T16:06:49.272127+0000","last_became_active":"2024-01-25T16:05:46.806972+0000","last_became_peered":"2024-01-25T16:05:46.806972+0000","last_unstale":"2024-01-25T16:06:49.272127+0000","last_undegraded":"2024-01-25T16:06:49.272127+0000","last_fullsized":"2024-01-25T16:06:49.272127+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T23:54:44.104250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.691434+0000","last_change":"2024-01-25T16:05:45.767147+0000","last_active":"2024-01-25T16:06:49.691434+0000","last_peered":"2024-01-25T16:06:49.691434+0000","last_clean":"2024-01-25T16:06:49.691434+0000","last_became_active":"2024-01-25T16:05:45.766811+0000","last_became_peered":"2024-01-25T16:05:45.766811+0000","last_unstale":"2024-01-25T16:06:49.691434+0000","last_undegraded":"2024-01-25T16:06:49.691434+0000","last_fullsized":"2024-01-25T16:06:49.691434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:51:59.916465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.144682+0000","last_change":"2024-01-25T16:06:50.144682+0000","last_active":"2024-01-25T16:06:50.144682+0000","last_peered":"2024-01-25T16:06:50.144682+0000","last_clean":"2024-01-25T16:06:50.144682+0000","last_became_active":"2024-01-25T16:06:50.144301+0000","last_became_peered":"2024-01-25T16:06:50.144301+0000","last_unstale":"2024-01-25T16:06:50.144682+0000","last_undegraded":"2024-01-25T16:06:50.144682+0000","last_fullsized":"2024-01-25T16:06:50.144682+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T19:14:24.797101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866249+0000","last_change":"2024-01-25T16:05:45.761456+0000","last_active":"2024-01-25T16:06:49.866249+0000","last_peered":"2024-01-25T16:06:49.866249+0000","last_clean":"2024-01-25T16:06:49.866249+0000","last_became_active":"2024-01-25T16:05:45.761127+0000","last_became_peered":"2024-01-25T16:05:45.761127+0000","last_unstale":"2024-01-25T16:06:49.866249+0000","last_undegraded":"2024-01-25T16:06:49.866249+0000","last_fullsized":"2024-01-25T16:06:49.866249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:44:00.091230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274600+0000","last_change":"2024-01-25T16:05:46.792572+0000","last_active":"2024-01-25T16:06:49.274600+0000","last_peered":"2024-01-25T16:06:49.274600+0000","last_clean":"2024-01-25T16:06:49.274600+0000","last_became_active":"2024-01-25T16:05:46.792201+0000","last_became_peered":"2024-01-25T16:05:46.792201+0000","last_unstale":"2024-01-25T16:06:49.274600+0000","last_undegraded":"2024-01-25T16:06:49.274600+0000","last_fullsized":"2024-01-25T16:06:49.274600+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:17:29.636840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271601+0000","last_change":"2024-01-25T16:05:45.770183+0000","last_active":"2024-01-25T16:06:49.271601+0000","last_peered":"2024-01-25T16:06:49.271601+0000","last_clean":"2024-01-25T16:06:49.271601+0000","last_became_active":"2024-01-25T16:05:45.769573+0000","last_became_peered":"2024-01-25T16:05:45.769573+0000","last_unstale":"2024-01-25T16:06:49.271601+0000","last_undegraded":"2024-01-25T16:06:49.271601+0000","last_fullsized":"2024-01-25T16:06:49.271601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:22:39.522177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.267878+0000","last_change":"2024-01-25T16:05:46.795137+0000","last_active":"2024-01-25T16:06:49.267878+0000","last_peered":"2024-01-25T16:06:49.267878+0000","last_clean":"2024-01-25T16:06:49.267878+0000","last_became_active":"2024-01-25T16:05:46.794836+0000","last_became_peered":"2024-01-25T16:05:46.794836+0000","last_unstale":"2024-01-25T16:06:49.267878+0000","last_undegraded":"2024-01-25T16:06:49.267878+0000","last_fullsized":"2024-01-25T16:06:49.267878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:51:27.421203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270539+0000","last_change":"2024-01-25T16:05:45.765813+0000","last_active":"2024-01-25T16:06:49.270539+0000","last_peered":"2024-01-25T16:06:49.270539+0000","last_clean":"2024-01-25T16:06:49.270539+0000","last_became_active":"2024-01-25T16:05:45.765520+0000","last_became_peered":"2024-01-25T16:05:45.765520+0000","last_unstale":"2024-01-25T16:06:49.270539+0000","last_undegraded":"2024-01-25T16:06:49.270539+0000","last_fullsized":"2024-01-25T16:06:49.270539+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:39:45.011355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272666+0000","last_change":"2024-01-25T16:05:46.805665+0000","last_active":"2024-01-25T16:06:49.272666+0000","last_peered":"2024-01-25T16:06:49.272666+0000","last_clean":"2024-01-25T16:06:49.272666+0000","last_became_active":"2024-01-25T16:05:46.804989+0000","last_became_peered":"2024-01-25T16:05:46.804989+0000","last_unstale":"2024-01-25T16:06:49.272666+0000","last_undegraded":"2024-01-25T16:06:49.272666+0000","last_fullsized":"2024-01-25T16:06:49.272666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:35:07.321023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.274825+0000","last_change":"2024-01-25T16:05:45.769982+0000","last_active":"2024-01-25T16:06:49.274825+0000","last_peered":"2024-01-25T16:06:49.274825+0000","last_clean":"2024-01-25T16:06:49.274825+0000","last_became_active":"2024-01-25T16:05:45.769617+0000","last_became_peered":"2024-01-25T16:05:45.769617+0000","last_unstale":"2024-01-25T16:06:49.274825+0000","last_undegraded":"2024-01-25T16:06:49.274825+0000","last_fullsized":"2024-01-25T16:06:49.274825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:07:23.225258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568330+0000","last_change":"2024-01-25T16:05:46.793665+0000","last_active":"2024-01-25T16:06:49.568330+0000","last_peered":"2024-01-25T16:06:49.568330+0000","last_clean":"2024-01-25T16:06:49.568330+0000","last_became_active":"2024-01-25T16:05:46.793335+0000","last_became_peered":"2024-01-25T16:05:46.793335+0000","last_unstale":"2024-01-25T16:06:49.568330+0000","last_undegraded":"2024-01-25T16:06:49.568330+0000","last_fullsized":"2024-01-25T16:06:49.568330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:43:51.216235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271009+0000","last_change":"2024-01-25T16:05:45.768136+0000","last_active":"2024-01-25T16:06:49.271009+0000","last_peered":"2024-01-25T16:06:49.271009+0000","last_clean":"2024-01-25T16:06:49.271009+0000","last_became_active":"2024-01-25T16:05:45.767847+0000","last_became_peered":"2024-01-25T16:05:45.767847+0000","last_unstale":"2024-01-25T16:06:49.271009+0000","last_undegraded":"2024-01-25T16:06:49.271009+0000","last_fullsized":"2024-01-25T16:06:49.271009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:11:16.154786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568231+0000","last_change":"2024-01-25T16:05:46.794454+0000","last_active":"2024-01-25T16:06:49.568231+0000","last_peered":"2024-01-25T16:06:49.568231+0000","last_clean":"2024-01-25T16:06:49.568231+0000","last_became_active":"2024-01-25T16:05:46.794135+0000","last_became_peered":"2024-01-25T16:05:46.794135+0000","last_unstale":"2024-01-25T16:06:49.568231+0000","last_undegraded":"2024-01-25T16:06:49.568231+0000","last_fullsized":"2024-01-25T16:06:49.568231+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T00:17:56.406013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556693+0000","last_change":"2024-01-25T16:05:45.763459+0000","last_active":"2024-01-25T16:06:49.556693+0000","last_peered":"2024-01-25T16:06:49.556693+0000","last_clean":"2024-01-25T16:06:49.556693+0000","last_became_active":"2024-01-25T16:05:45.763142+0000","last_became_peered":"2024-01-25T16:05:45.763142+0000","last_unstale":"2024-01-25T16:06:49.556693+0000","last_undegraded":"2024-01-25T16:06:49.556693+0000","last_fullsized":"2024-01-25T16:06:49.556693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:52:59.970813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568059+0000","last_change":"2024-01-25T16:05:46.797135+0000","last_active":"2024-01-25T16:06:49.568059+0000","last_peered":"2024-01-25T16:06:49.568059+0000","last_clean":"2024-01-25T16:06:49.568059+0000","last_became_active":"2024-01-25T16:05:46.796838+0000","last_became_peered":"2024-01-25T16:05:46.796838+0000","last_unstale":"2024-01-25T16:06:49.568059+0000","last_undegraded":"2024-01-25T16:06:49.568059+0000","last_fullsized":"2024-01-25T16:06:49.568059+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:36:59.271257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143019+0000","last_change":"2024-01-25T16:05:45.762903+0000","last_active":"2024-01-25T16:06:50.143019+0000","last_peered":"2024-01-25T16:06:50.143019+0000","last_clean":"2024-01-25T16:06:50.143019+0000","last_became_active":"2024-01-25T16:05:45.762559+0000","last_became_peered":"2024-01-25T16:05:45.762559+0000","last_unstale":"2024-01-25T16:06:50.143019+0000","last_undegraded":"2024-01-25T16:06:50.143019+0000","last_fullsized":"2024-01-25T16:06:50.143019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:44.743372+0000","last_clean_scrub_stamp":"2024-01-25T16:05:44.743372+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:58:57.511914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.568299+0000","last_change":"2024-01-25T16:05:46.792890+0000","last_active":"2024-01-25T16:06:49.568299+0000","last_peered":"2024-01-25T16:06:49.568299+0000","last_clean":"2024-01-25T16:06:49.568299+0000","last_became_active":"2024-01-25T16:05:46.792561+0000","last_became_peered":"2024-01-25T16:05:46.792561+0000","last_unstale":"2024-01-25T16:06:49.568299+0000","last_undegraded":"2024-01-25T16:06:49.568299+0000","last_fullsized":"2024-01-25T16:06:49.568299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T17:55:20.348743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.272334+0000","last_change":"2024-01-25T16:06:49.272334+0000","last_active":"2024-01-25T16:06:49.272334+0000","last_peered":"2024-01-25T16:06:49.272334+0000","last_clean":"2024-01-25T16:06:49.272334+0000","last_became_active":"2024-01-25T16:06:49.271856+0000","last_became_peered":"2024-01-25T16:06:49.271856+0000","last_unstale":"2024-01-25T16:06:49.272334+0000","last_undegraded":"2024-01-25T16:06:49.272334+0000","last_fullsized":"2024-01-25T16:06:49.272334+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T20:11:46.364112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.692112+0000","last_change":"2024-01-25T16:06:49.692112+0000","last_active":"2024-01-25T16:06:49.692112+0000","last_peered":"2024-01-25T16:06:49.692112+0000","last_clean":"2024-01-25T16:06:49.692112+0000","last_became_active":"2024-01-25T16:06:49.691777+0000","last_became_peered":"2024-01-25T16:06:49.691777+0000","last_unstale":"2024-01-25T16:06:49.692112+0000","last_undegraded":"2024-01-25T16:06:49.692112+0000","last_fullsized":"2024-01-25T16:06:49.692112+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:02:14.205358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.270501+0000","last_change":"2024-01-25T16:05:46.797036+0000","last_active":"2024-01-25T16:06:49.270501+0000","last_peered":"2024-01-25T16:06:49.270501+0000","last_clean":"2024-01-25T16:06:49.270501+0000","last_became_active":"2024-01-25T16:05:46.796703+0000","last_became_peered":"2024-01-25T16:05:46.796703+0000","last_unstale":"2024-01-25T16:06:49.270501+0000","last_undegraded":"2024-01-25T16:06:49.270501+0000","last_fullsized":"2024-01-25T16:06:49.270501+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T18:06:00.095666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.273783+0000","last_change":"2024-01-25T16:05:46.796367+0000","last_active":"2024-01-25T16:06:49.273783+0000","last_peered":"2024-01-25T16:06:49.273783+0000","last_clean":"2024-01-25T16:06:49.273783+0000","last_became_active":"2024-01-25T16:05:46.795390+0000","last_became_peered":"2024-01-25T16:05:46.795390+0000","last_unstale":"2024-01-25T16:06:49.273783+0000","last_undegraded":"2024-01-25T16:06:49.273783+0000","last_fullsized":"2024-01-25T16:06:49.273783+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:35:41.524542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143160+0000","last_change":"2024-01-25T16:05:46.796127+0000","last_active":"2024-01-25T16:06:50.143160+0000","last_peered":"2024-01-25T16:06:50.143160+0000","last_clean":"2024-01-25T16:06:50.143160+0000","last_became_active":"2024-01-25T16:05:46.795667+0000","last_became_peered":"2024-01-25T16:05:46.795667+0000","last_unstale":"2024-01-25T16:06:50.143160+0000","last_undegraded":"2024-01-25T16:06:50.143160+0000","last_fullsized":"2024-01-25T16:06:50.143160+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:00:44.411676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.268030+0000","last_change":"2024-01-25T16:05:46.796491+0000","last_active":"2024-01-25T16:06:49.268030+0000","last_peered":"2024-01-25T16:06:49.268030+0000","last_clean":"2024-01-25T16:06:49.268030+0000","last_became_active":"2024-01-25T16:05:46.796201+0000","last_became_peered":"2024-01-25T16:05:46.796201+0000","last_unstale":"2024-01-25T16:06:49.268030+0000","last_undegraded":"2024-01-25T16:06:49.268030+0000","last_fullsized":"2024-01-25T16:06:49.268030+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T01:39:42.396263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271125+0000","last_change":"2024-01-25T16:05:46.790680+0000","last_active":"2024-01-25T16:06:49.271125+0000","last_peered":"2024-01-25T16:06:49.271125+0000","last_clean":"2024-01-25T16:06:49.271125+0000","last_became_active":"2024-01-25T16:05:46.790330+0000","last_became_peered":"2024-01-25T16:05:46.790330+0000","last_unstale":"2024-01-25T16:06:49.271125+0000","last_undegraded":"2024-01-25T16:06:49.271125+0000","last_fullsized":"2024-01-25T16:06:49.271125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T21:22:33.452484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271871+0000","last_change":"2024-01-25T16:05:46.802958+0000","last_active":"2024-01-25T16:06:49.271871+0000","last_peered":"2024-01-25T16:06:49.271871+0000","last_clean":"2024-01-25T16:06:49.271871+0000","last_became_active":"2024-01-25T16:05:46.802606+0000","last_became_peered":"2024-01-25T16:05:46.802606+0000","last_unstale":"2024-01-25T16:06:49.271871+0000","last_undegraded":"2024-01-25T16:06:49.271871+0000","last_fullsized":"2024-01-25T16:06:49.271871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T16:57:44.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.556353+0000","last_change":"2024-01-25T16:05:46.794974+0000","last_active":"2024-01-25T16:06:49.556353+0000","last_peered":"2024-01-25T16:06:49.556353+0000","last_clean":"2024-01-25T16:06:49.556353+0000","last_became_active":"2024-01-25T16:05:46.794629+0000","last_became_peered":"2024-01-25T16:05:46.794629+0000","last_unstale":"2024-01-25T16:06:49.556353+0000","last_undegraded":"2024-01-25T16:06:49.556353+0000","last_fullsized":"2024-01-25T16:06:49.556353+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:19:22.918160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.271462+0000","last_change":"2024-01-25T16:05:46.798725+0000","last_active":"2024-01-25T16:06:49.271462+0000","last_peered":"2024-01-25T16:06:49.271462+0000","last_clean":"2024-01-25T16:06:49.271462+0000","last_became_active":"2024-01-25T16:05:46.798373+0000","last_became_peered":"2024-01-25T16:05:46.798373+0000","last_unstale":"2024-01-25T16:06:49.271462+0000","last_undegraded":"2024-01-25T16:06:49.271462+0000","last_fullsized":"2024-01-25T16:06:49.271462+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:55:31.117988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.866545+0000","last_change":"2024-01-25T16:05:46.795421+0000","last_active":"2024-01-25T16:06:49.866545+0000","last_peered":"2024-01-25T16:06:49.866545+0000","last_clean":"2024-01-25T16:06:49.866545+0000","last_became_active":"2024-01-25T16:05:46.795068+0000","last_became_peered":"2024-01-25T16:05:46.795068+0000","last_unstale":"2024-01-25T16:06:49.866545+0000","last_undegraded":"2024-01-25T16:06:49.866545+0000","last_fullsized":"2024-01-25T16:06:49.866545+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T03:20:46.490531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:50.143322+0000","last_change":"2024-01-25T16:05:46.802408+0000","last_active":"2024-01-25T16:06:50.143322+0000","last_peered":"2024-01-25T16:06:50.143322+0000","last_clean":"2024-01-25T16:06:50.143322+0000","last_became_active":"2024-01-25T16:05:46.802104+0000","last_became_peered":"2024-01-25T16:05:46.802104+0000","last_unstale":"2024-01-25T16:06:50.143322+0000","last_undegraded":"2024-01-25T16:06:50.143322+0000","last_fullsized":"2024-01-25T16:06:50.143322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-27T02:00:11.366282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-01-25T16:06:49.276111+0000","last_change":"2024-01-25T16:06:49.276111+0000","last_active":"2024-01-25T16:06:49.276111+0000","last_peered":"2024-01-25T16:06:49.276111+0000","last_clean":"2024-01-25T16:06:49.276111+0000","last_became_active":"2024-01-25T16:06:49.275665+0000","last_became_peered":"2024-01-25T16:06:49.275665+0000","last_unstale":"2024-01-25T16:06:49.276111+0000","last_undegraded":"2024-01-25T16:06:49.276111+0000","last_fullsized":"2024-01-25T16:06:49.276111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-25T16:05:45.747044+0000","last_clean_scrub_stamp":"2024-01-25T16:05:45.747044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-26T22:09:56.382891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60631,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":455,"num_read_kb":291,"num_write":152,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":34,"num_keys_recovered":17,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":181893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131,"ondisk_log_size":131,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":460341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.52800000000000002,"15min":0.53400000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.70399999999999996,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.4710000000000001,"5min":0.71199999999999997,"15min":0.58599999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":15.404999999999999,"5min":15.404999999999999,"15min":15.404999999999999},"last":0.61099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.63100000000000001,"15min":0.621},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.47699999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.70299999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.45900000000000002,"15min":0.44400000000000001},"min":{"1min":0.38100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.47499999999999998,"15min":0.46400000000000002},"min":{"1min":0.32500000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":4,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.54000000000000004,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.65100000000000002,"15min":0.64900000000000002},"min":{"1min":0.53900000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.81499999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54700000000000004,"15min":0.54500000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76200000000000001,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.60399999999999998,"15min":0.59599999999999997},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.84899999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:05:56 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.45000000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.71999999999999997,"15min":0.71599999999999997},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.879,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.73399999999999999,"15min":0.72599999999999998},"min":{"1min":0.51100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.90100000000000002,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.79500000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.748},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.81499999999999995}]},{"osd":9,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.91500000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.73199999999999998,"15min":0.71799999999999997},"min":{"1min":0.46899999999999997,"5min":0.44,"15min":0.44},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Jan 25 16:06:57 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73999999999999999,"15min":0.73599999999999999},"min":{"1min":0.68300000000000005,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.93899999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77500000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.94199999999999995,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.84199999999999997}]}]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1654784,"data_stored":1108419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:50 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.438}]},{"osd":1,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46700000000000003,"15min":0.47099999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.81699999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.64400000000000002,"15min":0.64200000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64600000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.82699999999999996,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.629}]},{"osd":5,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.81200000000000006,"5min":1.171,"15min":1.171},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.93000000000000005,"5min":1.139,"15min":1.139},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.75900000000000001,"15min":0.76900000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.1639999999999999,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Thu Jan 25 16:05:33 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.84499999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.96999999999999997,"5min":1.272,"15min":1.272},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.82499999999999996,"15min":0.84099999999999997},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98399999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.81899999999999995,"15min":0.83099999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80900000000000005,"15min":0.82899999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92600000000000005,"5min":1.331,"15min":1.331},"last":0.81000000000000005}]},{"osd":11,"last update":"Thu Jan 25 16:06:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.78700000000000003,"15min":0.79000000000000004},"min":{"1min":0.61599999999999999,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79300000000000004,"15min":0.78700000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.872}]}]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":460543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45900000000000002,"15min":0.46999999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.53400000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.498,"15min":0.51000000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68600000000000005,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.375}]},{"osd":1,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.439,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79600000000000004,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.625},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.748,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.45100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77900000000000003,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.60899999999999999,"15min":0.625},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63300000000000001,"5min":0.997,"15min":0.997},"last":0.56799999999999995}]},{"osd":3,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.54800000000000004,"15min":0.56699999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.59099999999999997,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.63,"15min":0.65400000000000003},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.70799999999999996,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":1.413,"5min":0.79700000000000004,"15min":0.69399999999999995},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":17.481999999999999,"5min":17.481999999999999,"15min":17.481999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.69299999999999995,"15min":0.71099999999999997},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.78600000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.81000000000000005,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.4219999999999999,"5min":0.93100000000000005,"15min":0.84899999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":16.093,"5min":16.093,"15min":16.093},"last":0.64500000000000002}]},{"osd":6,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82699999999999996,"5min":1.03,"15min":1.03},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.67000000000000004,"15min":0.69099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86099999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.51900000000000002}]},{"osd":7,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.80800000000000005,"15min":0.83099999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88800000000000001,"5min":1.167,"15min":1.167},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.84599999999999997,"15min":0.872},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91200000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.67300000000000004}]},{"osd":8,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":0.93000000000000005,"15min":0.84699999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":16.106000000000002,"5min":16.106000000000002,"15min":16.106000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.80700000000000005,"15min":0.82899999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.89600000000000002,"5min":1.23,"15min":1.23},"last":0.63300000000000001}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.72699999999999998,"15min":0.73899999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":0.80900000000000005,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.5960000000000001,"5min":0.94299999999999995,"15min":0.83399999999999996},"min":{"1min":0.61599999999999999,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":17.495000000000001,"5min":17.495000000000001,"15min":17.495000000000001},"last":0.70099999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":460463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.35899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.53700000000000003},"min":{"1min":0.39300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.68300000000000005}]},{"osd":1,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.59299999999999997,"15min":0.57899999999999996},"min":{"1min":0.47699999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.45800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.878}]},{"osd":2,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.46500000000000002,"15min":0.45400000000000001},"min":{"1min":0.33700000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.44400000000000001,"15min":0.42999999999999999},"min":{"1min":0.30599999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53800000000000003}]},{"osd":3,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.59099999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.70599999999999996}]},{"osd":4,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.61199999999999999,"15min":0.59499999999999997},"min":{"1min":0.40799999999999997,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.747},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.61899999999999999,"15min":0.60399999999999998},"min":{"1min":0.438,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.89700000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53500000000000003,"15min":0.52800000000000002},"min":{"1min":0.37,"5min":0.309,"15min":0.309},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.55000000000000004,"15min":0.54200000000000004},"min":{"1min":0.42899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.78800000000000003}]},{"osd":6,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.63,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.44900000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.83499999999999996}]},{"osd":7,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.59099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.69799999999999995,"15min":0.68799999999999994},"min":{"1min":0.52400000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.86699999999999999}]},{"osd":9,"last update":"Thu Jan 25 16:06:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69599999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.55100000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.95599999999999996,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.92100000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:36 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.71899999999999997,"15min":0.69999999999999996},"min":{"1min":0.52300000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.71799999999999997,"15min":0.70199999999999996},"min":{"1min":0.54400000000000004,"5min":0.495,"15min":0.495},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.94499999999999995}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.98799999999999999}]}]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32720,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712432,"statfs":{"total":95995035648,"available":95961530368,"internally_reserved":0,"allocated":1572864,"data_stored":1052392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:21 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.495,"15min":0.47699999999999998},"min":{"1min":0.44800000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.39700000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.63100000000000001}]},{"osd":1,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.28299999999999997,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.434,"15min":0.40600000000000003},"min":{"1min":0.313,"5min":0.247,"15min":0.247},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":0.51900000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.39400000000000002,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.59599999999999997,"15min":0.58099999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.56000000000000005}]},{"osd":3,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.57199999999999995,"15min":0.53400000000000003},"min":{"1min":0.36599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.3610000000000002,"5min":3.3610000000000002,"15min":3.3610000000000002},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.61699999999999999,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":4.8920000000000003,"5min":4.8920000000000003,"15min":4.8920000000000003},"last":0.69099999999999995}]},{"osd":4,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.53000000000000003,"15min":0.497},"min":{"1min":0.32400000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":3.2389999999999999,"5min":3.2389999999999999,"15min":3.2389999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.51100000000000001,"15min":0.5},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":0.49399999999999999}]},{"osd":5,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.61299999999999999,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.63,"15min":0.59999999999999998},"min":{"1min":0.439,"5min":0.433,"15min":0.433},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.66700000000000004}]},{"osd":6,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.70899999999999996,"15min":0.65100000000000002},"min":{"1min":0.55500000000000005,"5min":0.443,"15min":0.443},"max":{"1min":5.931,"5min":5.931,"15min":5.931},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.64500000000000002,"15min":0.628},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.3330000000000002,"5min":2.3330000000000002,"15min":2.3330000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:06:41 2024","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.68600000000000005,"15min":0.65500000000000003},"min":{"1min":0.58199999999999996,"5min":0.432,"15min":0.432},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.73899999999999999,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":1.9419999999999999},"last":0.80300000000000005}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.75600000000000001,"15min":0.751},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.78300000000000003,"15min":0.75900000000000001},"min":{"1min":0.53800000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":5.1269999999999998,"5min":5.1269999999999998,"15min":5.1269999999999998},"last":0.79300000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:30 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.72899999999999998,"15min":0.72199999999999998},"min":{"1min":0.47499999999999998,"5min":0.435,"15min":0.435},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.75700000000000001,"15min":0.748},"min":{"1min":0.44,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.9980000000000002,"5min":2.9980000000000002,"15min":2.9980000000000002},"last":0.77300000000000002}]},{"osd":11,"last update":"Thu Jan 25 16:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.81399999999999995}]}]},{"osd":6,"up_from":47,"seq":201863462966,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":459932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.42299999999999999,"15min":0.41199999999999998},"min":{"1min":0.26400000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60099999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.501},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45200000000000001,"15min":0.42899999999999999},"min":{"1min":0.26900000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":0.77300000000000002,"15min":0.66900000000000004},"min":{"1min":0.47699999999999998,"5min":0.37,"15min":0.37},"max":{"1min":12.151,"5min":12.151,"15min":12.151},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65200000000000002,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.57499999999999996,"15min":0.55600000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":1,"15min":1},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84199999999999997,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.57099999999999995}]},{"osd":3,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.50900000000000001,"15min":0.50800000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.64700000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54100000000000004,"15min":0.52500000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.718,"5min":1.718,"15min":1.718},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61299999999999999,"15min":0.59899999999999998},"min":{"1min":0.48599999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83099999999999996,"5min":1.131,"15min":1.131},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.69899999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:07:00 2024","interfaces":[{"interface":"back","average":{"1min":1.7370000000000001,"5min":0.88300000000000001,"15min":0.71299999999999997},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":19.369,"5min":19.369,"15min":19.369},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.40500000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.83099999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.40500000000000003}]},{"osd":7,"last update":"Thu Jan 25 16:06:18 2024","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73399999999999999,"15min":0.71199999999999997},"min":{"1min":0.51500000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.75600000000000001,"15min":0.746},"min":{"1min":0.55800000000000005,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0169999999999999,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.75600000000000001,"15min":0.76600000000000001},"min":{"1min":0.48799999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.89600000000000002,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":1.931,"5min":1.0109999999999999,"15min":0.85499999999999998},"min":{"1min":0.58599999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":19.265999999999998,"5min":19.265999999999998,"15min":19.265999999999998},"last":0.71499999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.74399999999999999}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":1.891,"5min":1.0429999999999999,"15min":0.90100000000000002},"min":{"1min":0.54300000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":19.245000000000001,"5min":19.245000000000001,"15min":19.245000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.893,"5min":1.085,"15min":0.95099999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":19.334,"5min":19.334,"15min":19.334},"last":0.79600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":516356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.49299999999999999},"min":{"1min":0.34599999999999997,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85099999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.51100000000000001,"15min":0.48599999999999999},"min":{"1min":0.30599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67600000000000005}]},{"osd":2,"last update":"Thu Jan 25 16:06:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.58699999999999997,"15min":0.55900000000000005},"min":{"1min":0.41599999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.873,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.437,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82499999999999996,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.378}]},{"osd":3,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58599999999999997,"15min":0.55300000000000005},"min":{"1min":0.435,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.61499999999999999,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.747}]},{"osd":4,"last update":"Thu Jan 25 16:06:13 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.42599999999999999},"min":{"1min":0.34499999999999997,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.871,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.45900000000000002,"15min":0.442},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.81899999999999995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.63400000000000001,"15min":0.58799999999999997},"min":{"1min":0.55300000000000005,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.98799999999999999,"5min":1.109,"15min":1.109},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.61699999999999999,"15min":0.58299999999999996},"min":{"1min":0.52300000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.91900000000000004,"5min":1.002,"15min":1.002},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:06:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.627,"15min":0.60699999999999998},"min":{"1min":0.47699999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.96499999999999997,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.63700000000000001,"15min":0.59999999999999998},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.0680000000000001,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.56999999999999995}]},{"osd":7,"last update":"Thu Jan 25 16:06:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.57099999999999995,"15min":0.54300000000000004},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.97199999999999998,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.436},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.57899999999999996,"15min":0.55400000000000005},"min":{"1min":0.40600000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98799999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.73699999999999999,"15min":0.71699999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.048,"5min":1.155,"15min":1.155},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.74399999999999999,"15min":0.70999999999999996},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.626}]},{"osd":9,"last update":"Thu Jan 25 16:06:07 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67900000000000005,"15min":0.67100000000000004},"min":{"1min":0.55500000000000005,"5min":0.501,"15min":0.501},"max":{"1min":1.046,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.77600000000000002,"15min":0.77000000000000002},"min":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0469999999999999,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.77300000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80000000000000004,"15min":0.79400000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.089,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.59099999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.016,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59599999999999997}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69099999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":461203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 25 16:06:02 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.53400000000000003,"15min":0.51900000000000002},"min":{"1min":0.31900000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.496},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.52300000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.878,"5min":0.996,"15min":0.996},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 25 16:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56299999999999994,"15min":0.52300000000000002},"min":{"1min":0.40799999999999997,"5min":0.313,"15min":0.313},"max":{"1min":0.68400000000000005,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.56399999999999995,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.308,"15min":0.308},"max":{"1min":0.77700000000000002,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 25 16:06:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.35799999999999998}]},{"osd":4,"last update":"Thu Jan 25 16:06:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56100000000000005,"15min":0.53300000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.54500000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75,"5min":1.012,"15min":1.012},"last":0.40000000000000002}]},{"osd":5,"last update":"Thu Jan 25 16:06:37 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56399999999999995,"15min":0.53600000000000003},"min":{"1min":0.434,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.70599999999999996,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60099999999999998,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.79900000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.51800000000000002,"15min":0.52700000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81399999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.432},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71499999999999997,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 25 16:05:19 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68700000000000006,"15min":0.68100000000000005},"min":{"1min":0.52100000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.88900000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.45200000000000001}]},{"osd":8,"last update":"Thu Jan 25 16:06:42 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.746,"15min":0.77500000000000002},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.78700000000000003,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.75600000000000001,"15min":0.78600000000000003},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.77200000000000002,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53800000000000003}]},{"osd":9,"last update":"Thu Jan 25 16:06:06 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.71699999999999997,"15min":0.74099999999999999},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88900000000000001,"5min":1.115,"15min":1.115},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.71599999999999997,"15min":0.73699999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.92400000000000004,"5min":1.036,"15min":1.036},"last":0.71699999999999997}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.79200000000000004,"15min":0.81100000000000005},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.81799999999999995,"5min":1.107,"15min":1.107},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.77600000000000002,"15min":0.79400000000000004},"min":{"1min":0.48799999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90300000000000002,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63200000000000001}]},{"osd":11,"last update":"Thu Jan 25 16:06:58 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.58799999999999997,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.83299999999999996,"5min":1.083,"15min":1.083},"last":0.753},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73199999999999998,"15min":0.73299999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95599999999999996,"5min":0.999,"15min":0.999},"last":0.68799999999999994}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":518062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.52900000000000003,"15min":0.51900000000000002},"min":{"1min":0.36699999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51200000000000001,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 25 16:06:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.52700000000000002},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.83999999999999997,"5min":0.873,"15min":0.873},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.57899999999999996,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.876,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.66600000000000004}]},{"osd":3,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58799999999999997,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83599999999999997,"5min":0.871,"15min":0.871},"last":0.47999999999999998},{"interface":"front","average":{"1min":1.3360000000000001,"5min":0.74299999999999999,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":16.850999999999999,"5min":16.850999999999999,"15min":16.850999999999999},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 25 16:06:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.46000000000000002,"15min":0.42799999999999999},"min":{"1min":0.32400000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.45000000000000001},"min":{"1min":0.40799999999999997,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.32700000000000001}]},{"osd":6,"last update":"Thu Jan 25 16:06:58 2024","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.65100000000000002,"15min":0.629},"min":{"1min":0.56499999999999995,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.83999999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.462,"5min":0.80400000000000005,"15min":0.67800000000000005},"min":{"1min":0.44600000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":16.923999999999999,"5min":16.923999999999999,"15min":16.923999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Jan 25 16:05:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64000000000000001,"15min":0.63800000000000001},"min":{"1min":0.44900000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.67500000000000004},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.499}]},{"osd":8,"last update":"Thu Jan 25 16:06:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.59899999999999998,"15min":0.59399999999999997},"min":{"1min":0.46300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.68100000000000005}]},{"osd":9,"last update":"Thu Jan 25 16:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.70699999999999996,"15min":0.69599999999999995},"min":{"1min":0.59399999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93600000000000005,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.46000000000000002}]},{"osd":10,"last update":"Thu Jan 25 16:06:33 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.56699999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.67700000000000005,"15min":0.65900000000000003},"min":{"1min":0.52700000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.71199999999999997}]},{"osd":11,"last update":"Thu Jan 25 16:06:47 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.64300000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":460243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.34799999999999998,"15min":0.32600000000000001},"min":{"1min":0.251,"5min":0.193,"15min":0.193},"max":{"1min":0.65700000000000003,"5min":0.755,"15min":0.755},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.36599999999999999,"15min":0.34200000000000003},"min":{"1min":0.25800000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.65000000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.504}]},{"osd":1,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.53400000000000003},"min":{"1min":0.42899999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.51700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.69299999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:06:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.33400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.80600000000000005,"5min":0.877,"15min":0.877},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63600000000000001,"15min":0.57399999999999995},"min":{"1min":0.432,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.79900000000000004,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.54200000000000004,"15min":0.52400000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88100000000000001,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.56599999999999995,"15min":0.53000000000000003},"min":{"1min":0.45900000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.82499999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 25 16:06:35 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.40600000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.84599999999999997,"5min":1.014,"15min":1.014},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.84799999999999998,"5min":1,"15min":1},"last":0.86699999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:56 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.49299999999999999,"15min":0.50700000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76500000000000001,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.371,"5min":0.315,"15min":0.315},"max":{"1min":0.80400000000000005,"5min":0.87,"15min":0.87},"last":0.76400000000000001}]},{"osd":7,"last update":"Thu Jan 25 16:06:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.61299999999999999,"15min":0.60199999999999998},"min":{"1min":0.43099999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81599999999999995,"5min":0.879,"15min":0.879},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.42399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72099999999999997}]},{"osd":8,"last update":"Thu Jan 25 16:05:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.89300000000000002,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.71099999999999997,"15min":0.72499999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.63900000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65900000000000003,"15min":0.66700000000000004},"min":{"1min":0.41499999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88500000000000001,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.67100000000000004,"15min":0.68100000000000005},"min":{"1min":0.44,"5min":0.316,"15min":0.316},"max":{"1min":0.84399999999999997,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.82299999999999995}]},{"osd":10,"last update":"Thu Jan 25 16:06:32 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.52500000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.86599999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.71499999999999997,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.93799999999999994,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.79800000000000004}]},{"osd":11,"last update":"Thu Jan 25 16:06:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.88700000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":460369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:05:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.438,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.50800000000000001,"15min":0.496},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.376,"15min":0.35399999999999998},"min":{"1min":0.27600000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.40699999999999997,"15min":0.38200000000000001},"min":{"1min":0.29299999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.86099999999999999,"5min":0.877,"15min":0.877},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.55500000000000005,"15min":0.52900000000000003},"min":{"1min":0.41399999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.56499999999999995,"15min":0.53900000000000003},"min":{"1min":0.47899999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75600000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.56299999999999994,"15min":0.54000000000000004},"min":{"1min":0.38500000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.78600000000000003,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.94099999999999995}]},{"osd":5,"last update":"Thu Jan 25 16:06:40 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.61099999999999999,"15min":0.57199999999999995},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.55500000000000005},"min":{"1min":0.44900000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.82399999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 25 16:06:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61099999999999999,"15min":0.58399999999999996},"min":{"1min":0.48699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.878},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.46100000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.78900000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.89400000000000002}]},{"osd":7,"last update":"Thu Jan 25 16:06:19 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.36099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.38400000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.86299999999999999}]},{"osd":8,"last update":"Thu Jan 25 16:06:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66300000000000003,"15min":0.65600000000000003},"min":{"1min":0.53400000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.80600000000000005,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.55700000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89200000000000002,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.76000000000000001}]},{"osd":9,"last update":"Thu Jan 25 16:06:10 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.70499999999999996,"15min":0.70299999999999996},"min":{"1min":0.54800000000000004,"5min":0.438,"15min":0.438},"max":{"1min":0.95899999999999996,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.84599999999999997}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.754},"min":{"1min":0.53800000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.91400000000000003,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73499999999999999,"15min":0.72999999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95799999999999996,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.72899999999999998}]},{"osd":11,"last update":"Thu Jan 25 16:06:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79600000000000004}]}]},{"osd":5,"up_from":39,"seq":167503724603,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712424,"statfs":{"total":95995035648,"available":95961522176,"internally_reserved":0,"allocated":1585152,"data_stored":1052076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.52800000000000002,"15min":0.51900000000000002},"min":{"1min":0.41599999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.377,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.751,"15min":0.751},"last":0.43099999999999999}]},{"osd":1,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58599999999999997,"15min":0.57099999999999995},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.80400000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.59099999999999997,"15min":0.56100000000000005},"min":{"1min":0.52400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58499999999999996}]},{"osd":2,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42399999999999999,"15min":0.40600000000000003},"min":{"1min":0.32100000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.436,"15min":0.42099999999999999},"min":{"1min":0.33800000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.317}]},{"osd":3,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.55200000000000005,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56599999999999995,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.443}]},{"osd":4,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.58599999999999997},"min":{"1min":0.48499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61799999999999999,"15min":0.60699999999999998},"min":{"1min":0.54800000000000004,"5min":0.378,"15min":0.378},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 25 16:06:59 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.60299999999999998,"15min":0.58499999999999996},"min":{"1min":0.40100000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.622,"15min":0.60599999999999998},"min":{"1min":0.434,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.83099999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.48799999999999999}]},{"osd":7,"last update":"Thu Jan 25 16:06:16 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68000000000000005,"15min":0.67800000000000005},"min":{"1min":0.47999999999999998,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.53900000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60299999999999998}]},{"osd":8,"last update":"Thu Jan 25 16:06:39 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.89500000000000002,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.54100000000000004,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.47099999999999997}]},{"osd":9,"last update":"Thu Jan 25 16:06:11 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.51200000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.70999999999999996,"15min":0.70299999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.54300000000000004}]},{"osd":10,"last update":"Thu Jan 25 16:06:34 2024","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.70499999999999996,"15min":0.69199999999999995},"min":{"1min":0.61199999999999999,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.72899999999999998,"15min":0.71399999999999997},"min":{"1min":0.63600000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":0.97999999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.65300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-25T16:07:06.101 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-01-25T16:07:06.102 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-25T16:07:06.102 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-01-25T16:07:06.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph health --format=json 2024-01-25T16:07:06.348 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:07.384 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:07 smithi033 ceph-mon[31417]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:07.384 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:07 smithi033 ceph-mon[31417]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:07 smithi152 ceph-mon[36015]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:07 smithi152 ceph-mon[36015]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:07 smithi076 ceph-mon[35642]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-01-25T16:07:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:07 smithi076 ceph-mon[35642]: pgmap v327: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 -- 172.21.15.33:0/1810629506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa4100710 msgr2=0x7f9aa4100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 --2- 172.21.15.33:0/1810629506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa4100710 0x7f9aa4100b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009990 tx=0x7f9a8c009ca0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 -- 172.21.15.33:0/1810629506 shutdown_connections 2024-01-25T16:07:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 --2- 172.21.15.33:0/1810629506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41010d0 0x7f9aa410d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 --2- 172.21.15.33:0/1810629506 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa41066d0 0x7f9aa4106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 --2- 172.21.15.33:0/1810629506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa4100710 0x7f9aa4100b90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 -- 172.21.15.33:0/1810629506 >> 172.21.15.33:0/1810629506 conn(0x7f9aa4076700 msgr2=0x7f9aa4076b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 -- 172.21.15.33:0/1810629506 shutdown_connections 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 -- 172.21.15.33:0/1810629506 wait complete. 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.898+0000 7f9aa9264700 1 Processor -- start 2024-01-25T16:07:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 -- start start 2024-01-25T16:07:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 0x7f9aa41026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 0x7f9aa4103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa41a6070 con 0x7f9aa41010d0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa41a61e0 con 0x7f9aa41066d0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa9264700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa41a63a0 con 0x7f9aa4100710 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53962/0 (socket says 172.21.15.33:53962) 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 -- 172.21.15.33:0/4073931347 learned_addr learned my addr 172.21.15.33:0/4073931347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa2ffd700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 0x7f9aa41026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa37fe700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 0x7f9aa4103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 -- 172.21.15.33:0/4073931347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 msgr2=0x7f9aa41026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 0x7f9aa41026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 -- 172.21.15.33:0/4073931347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 msgr2=0x7f9aa4103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.899+0000 7f9aa27fc700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 0x7f9aa4103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa27fc700 1 -- 172.21.15.33:0/4073931347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c0095f0 con 0x7f9aa41010d0 2024-01-25T16:07:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa37fe700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 0x7f9aa4103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa2ffd700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 0x7f9aa41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa27fc700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098e0 tx=0x7f9a8c00f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c01d070 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a8c00fb90 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa41a6620 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c017650 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.900+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa41a6ac0 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.901+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa4101e00 con 0x7f9aa41010d0 2024-01-25T16:07:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.903+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f9a8c00fd00 con 0x7f9aa41010d0 2024-01-25T16:07:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.903+0000 7f9a9bfff700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 0x7f9a900a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.904+0000 7f9aa2ffd700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 0x7f9a900a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.904+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9a8c0ce2e0 con 0x7f9aa41010d0 2024-01-25T16:07:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.904+0000 7f9aa2ffd700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 0x7f9a900a38c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009d50 tx=0x7f9a9c009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:07.907+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f9a8c097690 con 0x7f9aa41010d0 2024-01-25T16:07:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.152+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9aa40684a0 con 0x7f9aa41010d0 2024-01-25T16:07:08.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.152+0000 7f9a9bfff700 1 -- 172.21.15.33:0/4073931347 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f9a8c09acb0 con 0x7f9aa41010d0 2024-01-25T16:07:08.156 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:08.156 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 msgr2=0x7f9a900a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 0x7f9a900a38c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009d50 tx=0x7f9a9c009040 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 msgr2=0x7f9aa4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0098e0 tx=0x7f9a8c00f720 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 shutdown_connections 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f9a900a1400 0x7f9a900a38c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9aa41066d0 0x7f9aa4103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9aa4100710 0x7f9aa41026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 --2- 172.21.15.33:0/4073931347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9aa41010d0 0x7f9aa4102c20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 >> 172.21.15.33:0/4073931347 conn(0x7f9aa4076700 msgr2=0x7f9aa4104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:08.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 shutdown_connections 2024-01-25T16:07:08.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.154+0000 7f9aa9264700 1 -- 172.21.15.33:0/4073931347 wait complete. 2024-01-25T16:07:08.513 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:08 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4073931347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-25T16:07:08.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:08 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4073931347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-25T16:07:08.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:08 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4073931347' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-01-25T16:07:08.834 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-01-25T16:07:08.834 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:08.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/459787581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 msgr2=0x7f0e34101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/459787581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34101280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009900 tx=0x7f0e2802bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/459787581 shutdown_connections 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/459787581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e341017c0 0x7f0e34108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/459787581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34101280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/459787581 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e340ffc10 0x7f0e34100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/459787581 >> 172.21.15.33:0/459787581 conn(0x7f0e340fb180 msgr2=0x7f0e340fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/459787581 shutdown_connections 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/459787581 wait complete. 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 Processor -- start 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- start start 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e340ffc10 0x7f0e3410a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 0x7f0e34109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e341097b0 con 0x7f0e34100e00 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e34109920 con 0x7f0e340ffc10 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.938+0000 7f0e3ae2b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e341a0e00 con 0x7f0e341017c0 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40878/0 (socket says 172.21.15.33:40878) 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 -- 172.21.15.33:0/3756266748 learned_addr learned my addr 172.21.15.33:0/3756266748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e38bc7640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e340ffc10 0x7f0e3410a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e393c8640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 0x7f0e34109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 -- 172.21.15.33:0/3756266748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 msgr2=0x7f0e34109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 0x7f0e34109050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 -- 172.21.15.33:0/3756266748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e340ffc10 msgr2=0x7f0e3410a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e340ffc10 0x7f0e3410a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:08.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 -- 172.21.15.33:0/3756266748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e28009560 con 0x7f0e34100e00 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e393c8640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 0x7f0e34109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e33fff640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009430 tx=0x7f0e28004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280043d0 con 0x7f0e34100e00 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e280054c0 con 0x7f0e34100e00 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e341a1050 con 0x7f0e34100e00 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003e80 con 0x7f0e34100e00 2024-01-25T16:07:08.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.939+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e341a15e0 con 0x7f0e34100e00 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.940+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e34106f60 con 0x7f0e34100e00 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.941+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f0e280035f0 con 0x7f0e34100e00 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.942+0000 7f0e31ffb640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 0x7f0e1c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.942+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0e280ef700 con 0x7f0e34100e00 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.942+0000 7f0e38bc7640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 0x7f0e1c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:08.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.942+0000 7f0e38bc7640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 0x7f0e1c0a3830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e34100c60 tx=0x7f0e2c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:08.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:08.945+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f0e280b8b30 con 0x7f0e34100e00 2024-01-25T16:07:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.126+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e3410c890 con 0x7f0e34100e00 2024-01-25T16:07:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.128+0000 7f0e31ffb640 1 -- 172.21.15.33:0/3756266748 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f0e280bc150 con 0x7f0e34100e00 2024-01-25T16:07:09.132 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:09.132 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 msgr2=0x7f0e1c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 0x7f0e1c0a3830 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0e34100c60 tx=0x7f0e2c00d040 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 msgr2=0x7f0e34108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0e28009430 tx=0x7f0e28004ee0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 shutdown_connections 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e341017c0 0x7f0e34109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f0e1c0a1370 0x7f0e1c0a3830 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e34100e00 0x7f0e34108b10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 --2- 172.21.15.33:0/3756266748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e340ffc10 0x7f0e3410a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 >> 172.21.15.33:0/3756266748 conn(0x7f0e340fb180 msgr2=0x7f0e34104130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 shutdown_connections 2024-01-25T16:07:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.131+0000 7f0e3ae2b640 1 -- 172.21.15.33:0/3756266748 wait complete. 2024-01-25T16:07:09.148 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:07:09.148 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:07:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 -- 172.21.15.33:0/4142795259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 msgr2=0x7f6868108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 --2- 172.21.15.33:0/4142795259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868108050 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6860009900 tx=0x7f686002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 -- 172.21.15.33:0/4142795259 shutdown_connections 2024-01-25T16:07:09.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 --2- 172.21.15.33:0/4142795259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868108050 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 --2- 172.21.15.33:0/4142795259 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6868100e00 0x7f6868101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 --2- 172.21.15.33:0/4142795259 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68680ffc10 0x7f6868100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 -- 172.21.15.33:0/4142795259 >> 172.21.15.33:0/4142795259 conn(0x7f68680fb1a0 msgr2=0x7f68680fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 -- 172.21.15.33:0/4142795259 shutdown_connections 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.250+0000 7f686da2b640 1 -- 172.21.15.33:0/4142795259 wait complete. 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 Processor -- start 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 -- start start 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680ffc10 0x7f6868103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6868100e00 0x7f68681021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868102e80 con 0x7f68681017c0 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6868102ff0 con 0x7f6868100e00 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f686da2b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68681a0cc0 con 0x7f68680ffc10 2024-01-25T16:07:09.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f68677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f68677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40890/0 (socket says 172.21.15.33:40890) 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f68677fe640 1 -- 172.21.15.33:0/2572546171 learned_addr learned my addr 172.21.15.33:0/2572546171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f6866ffd640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680ffc10 0x7f6868103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.251+0000 7f68667fc640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6868100e00 0x7f68681021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 -- 172.21.15.33:0/2572546171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680ffc10 msgr2=0x7f6868103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680ffc10 0x7f6868103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 -- 172.21.15.33:0/2572546171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6868100e00 msgr2=0x7f68681021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6868100e00 0x7f68681021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 -- 172.21.15.33:0/2572546171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6860009560 con 0x7f68681017c0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f68677fe640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f686002c610 tx=0x7f6860004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6860005000 con 0x7f68681017c0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6860005620 con 0x7f68681017c0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68681a0f10 con 0x7f68681017c0 2024-01-25T16:07:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686003e450 con 0x7f68681017c0 2024-01-25T16:07:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.252+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68681a1390 con 0x7f68681017c0 2024-01-25T16:07:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.253+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686810b4b0 con 0x7f68681017c0 2024-01-25T16:07:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.254+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f6860005370 con 0x7f68681017c0 2024-01-25T16:07:09.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.255+0000 7f686ca29640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 0x7f68540a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.255+0000 7f6866ffd640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 0x7f68540a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.255+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f68600ef880 con 0x7f68681017c0 2024-01-25T16:07:09.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.255+0000 7f6866ffd640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 0x7f68540a3830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6868100c60 tx=0x7f6858009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.258+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f68600b8cb0 con 0x7f68681017c0 2024-01-25T16:07:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.436+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f686810b670 con 0x7f68681017c0 2024-01-25T16:07:09.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.437+0000 7f686ca29640 1 -- 172.21.15.33:0/2572546171 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68600bc2d0 con 0x7f68681017c0 2024-01-25T16:07:09.440 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:07:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 msgr2=0x7f68540a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 0x7f68540a3830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6868100c60 tx=0x7f6858009670 comp rx=0 tx=0).stop 2024-01-25T16:07:09.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 msgr2=0x7f6868102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f686002c610 tx=0x7f6860004270 comp rx=0 tx=0).stop 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 shutdown_connections 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f68540a1370 0x7f68540a3830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68681017c0 0x7f6868102720 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6868100e00 0x7f68681021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 --2- 172.21.15.33:0/2572546171 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f68680ffc10 0x7f6868103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 >> 172.21.15.33:0/2572546171 conn(0x7f68680fb1a0 msgr2=0x7f68680fcc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 shutdown_connections 2024-01-25T16:07:09.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.439+0000 7f686da2b640 1 -- 172.21.15.33:0/2572546171 wait complete. 2024-01-25T16:07:09.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:09 smithi033 ceph-mon[31417]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:09.452 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:09 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3756266748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:09.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:07:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 -- 172.21.15.33:0/158906736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 msgr2=0x7fc3f4101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 --2- 172.21.15.33:0/158906736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4101120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4009ca0 tx=0x7fc3e402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 -- 172.21.15.33:0/158906736 shutdown_connections 2024-01-25T16:07:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 --2- 172.21.15.33:0/158906736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f41028b0 0x7fc3f4109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 --2- 172.21.15.33:0/158906736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 --2- 172.21.15.33:0/158906736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4101120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 -- 172.21.15.33:0/158906736 >> 172.21.15.33:0/158906736 conn(0x7fc3f40fc290 msgr2=0x7fc3f40fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 -- 172.21.15.33:0/158906736 shutdown_connections 2024-01-25T16:07:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.554+0000 7fc3f9015640 1 -- 172.21.15.33:0/158906736 wait complete. 2024-01-25T16:07:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 Processor -- start 2024-01-25T16:07:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 -- start start 2024-01-25T16:07:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 0x7fc3f419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419e820 con 0x7fc3f4100d00 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419e990 con 0x7fc3f4101ef0 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.555+0000 7fc3f9015640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419eb50 con 0x7fc3f41028b0 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40916/0 (socket says 172.21.15.33:40916) 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 -- 172.21.15.33:0/4172626109 learned_addr learned my addr 172.21.15.33:0/4172626109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f259c640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f359e640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 0x7fc3f419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 -- 172.21.15.33:0/4172626109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 msgr2=0x7fc3f419e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 0x7fc3f419e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 -- 172.21.15.33:0/4172626109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 msgr2=0x7fc3f4197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 -- 172.21.15.33:0/4172626109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4009900 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f259c640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f359e640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 0x7fc3f419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f2d9d640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000f80 tx=0x7fc3e40329c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4034e80 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f419edd0 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3e4003e30 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e403d3f0 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.556+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f419f1e0 con 0x7fc3f4100d00 2024-01-25T16:07:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.557+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f41081b0 con 0x7fc3f4100d00 2024-01-25T16:07:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.559+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fc3e403b040 con 0x7fc3f4100d00 2024-01-25T16:07:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.559+0000 7fc3ebfff640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 0x7fc3e00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.560+0000 7fc3f259c640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 0x7fc3e00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.560+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc3e40ef830 con 0x7fc3f4100d00 2024-01-25T16:07:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.560+0000 7fc3f259c640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 0x7fc3e00a3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009b40 tx=0x7fc3dc005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.563+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fc3e40b8b90 con 0x7fc3f4100d00 2024-01-25T16:07:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:09 smithi152 ceph-mon[36015]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:09 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3756266748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:09 smithi076 ceph-mon[35642]: pgmap v328: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:09 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3756266748' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:09.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.741+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3f410c720 con 0x7fc3f4100d00 2024-01-25T16:07:09.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.742+0000 7fc3ebfff640 1 -- 172.21.15.33:0/4172626109 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3e40baac0 con 0x7fc3f4100d00 2024-01-25T16:07:09.746 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:07:09.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 msgr2=0x7fc3e00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 0x7fc3e00a3830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009b40 tx=0x7fc3dc005e50 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 msgr2=0x7fc3f4196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4000f80 tx=0x7fc3e40329c0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 shutdown_connections 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3f41028b0 0x7fc3f419e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3f4101ef0 0x7fc3f4197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fc3e00a1370 0x7fc3e00a3830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 --2- 172.21.15.33:0/4172626109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3f4100d00 0x7fc3f4196d40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 >> 172.21.15.33:0/4172626109 conn(0x7fc3f40fc290 msgr2=0x7fc3f4105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 shutdown_connections 2024-01-25T16:07:09.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.744+0000 7fc3f9015640 1 -- 172.21.15.33:0/4172626109 wait complete. 2024-01-25T16:07:09.759 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:07:09.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1559320206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 msgr2=0x7f5c801064a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1559320206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c801064a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009900 tx=0x7f5c7402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1559320206 shutdown_connections 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1559320206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c801069e0 0x7f5c80108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1559320206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c801064a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1559320206 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c80101780 0x7f5c80103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1559320206 >> 172.21.15.33:0/1559320206 conn(0x7f5c800fb190 msgr2=0x7f5c800fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1559320206 shutdown_connections 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.860+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1559320206 wait complete. 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 Processor -- start 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 -- start start 2024-01-25T16:07:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c80101780 0x7f5c801010f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c801069e0 0x7f5c800ffc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801003d0 con 0x7f5c801040b0 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80100540 con 0x7f5c80101780 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c85ff5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801a0d50 con 0x7f5c801069e0 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c7effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c7effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40932/0 (socket says 172.21.15.33:40932) 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.861+0000 7f5c7effd640 1 -- 172.21.15.33:0/1281380908 learned_addr learned my addr 172.21.15.33:0/1281380908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7ffff640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c801069e0 0x7f5c800ffc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7f7fe640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c80101780 0x7f5c801010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 -- 172.21.15.33:0/1281380908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c801069e0 msgr2=0x7f5c800ffc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c801069e0 0x7f5c800ffc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 -- 172.21.15.33:0/1281380908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c80101780 msgr2=0x7f5c801010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c80101780 0x7f5c801010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 -- 172.21.15.33:0/1281380908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009560 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7effd640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c74036040 tx=0x7f5c74004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740042d0 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c74004430 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c801a0fa0 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74003c90 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.862+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c801a1420 con 0x7f5c801040b0 2024-01-25T16:07:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.863+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c800fcc80 con 0x7f5c801040b0 2024-01-25T16:07:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.864+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5c74005000 con 0x7f5c801040b0 2024-01-25T16:07:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.865+0000 7f5c7cff9640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 0x7f5c680a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.865+0000 7f5c7f7fe640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 0x7f5c680a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.865+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5c740eee80 con 0x7f5c801040b0 2024-01-25T16:07:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.865+0000 7f5c7f7fe640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 0x7f5c680a3830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009aa0 tx=0x7f5c70005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:09.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:09.869+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5c740b82b0 con 0x7f5c801040b0 2024-01-25T16:07:10.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.049+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c8002ca70 con 0x7f5c801040b0 2024-01-25T16:07:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.049+0000 7f5c7cff9640 1 -- 172.21.15.33:0/1281380908 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c740bb8d0 con 0x7f5c801040b0 2024-01-25T16:07:10.053 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 msgr2=0x7f5c680a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 0x7f5c680a3830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009aa0 tx=0x7f5c70005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 msgr2=0x7f5c800ff730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5c74036040 tx=0x7f5c74004dd0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 shutdown_connections 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5c801069e0 0x7f5c800ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5c680a1370 0x7f5c680a3830 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c801040b0 0x7f5c800ff730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 --2- 172.21.15.33:0/1281380908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c80101780 0x7f5c801010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 >> 172.21.15.33:0/1281380908 conn(0x7f5c800fb190 msgr2=0x7f5c801022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 shutdown_connections 2024-01-25T16:07:10.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.051+0000 7f5c85ff5640 1 -- 172.21.15.33:0/1281380908 wait complete. 2024-01-25T16:07:10.066 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:07:10.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 -- 172.21.15.33:0/1088954235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 msgr2=0x7f283c108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 --2- 172.21.15.33:0/1088954235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c108050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2834009ca0 tx=0x7f283402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 -- 172.21.15.33:0/1088954235 shutdown_connections 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 --2- 172.21.15.33:0/1088954235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c108050 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 --2- 172.21.15.33:0/1088954235 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 --2- 172.21.15.33:0/1088954235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 0x7f283c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 -- 172.21.15.33:0/1088954235 >> 172.21.15.33:0/1088954235 conn(0x7f283c0fb180 msgr2=0x7f283c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 -- 172.21.15.33:0/1088954235 shutdown_connections 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.166+0000 7f28441b0640 1 -- 172.21.15.33:0/1088954235 wait complete. 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 Processor -- start 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 -- start start 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 0x7f283c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a0b20 con 0x7f283c1017c0 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a0c90 con 0x7f283c0ffc10 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f28441b0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c1a0e50 con 0x7f283c100e00 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40948/0 (socket says 172.21.15.33:40948) 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 -- 172.21.15.33:0/555642286 learned_addr learned my addr 172.21.15.33:0/555642286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f2841f4c640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 0x7f283c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284174b640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 -- 172.21.15.33:0/555642286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 msgr2=0x7f283c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 -- 172.21.15.33:0/555642286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 msgr2=0x7f283c199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 0x7f283c199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284274d640 1 -- 172.21.15.33:0/555642286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009900 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.167+0000 7f284174b640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f284274d640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2834037040 tx=0x7f2834032a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834003970 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2834004040 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c1a10d0 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283403d650 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.168+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c1a1650 con 0x7f283c1017c0 2024-01-25T16:07:10.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.169+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c10b5c0 con 0x7f283c1017c0 2024-01-25T16:07:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.171+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f283403b040 con 0x7f283c1017c0 2024-01-25T16:07:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.171+0000 7f2832ffd640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 0x7f28280a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.171+0000 7f2841f4c640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 0x7f28280a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.171+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28340ef520 con 0x7f283c1017c0 2024-01-25T16:07:10.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.172+0000 7f2841f4c640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 0x7f28280a38b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f283c100c60 tx=0x7f2838008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.176+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f28340b8950 con 0x7f283c1017c0 2024-01-25T16:07:10.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:10 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2572546171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:10 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4172626109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.342 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:10 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1281380908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.394+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f283c1a1af0 con 0x7f283c1017c0 2024-01-25T16:07:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.395+0000 7f2832ffd640 1 -- 172.21.15.33:0/555642286 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f28340bbf70 con 0x7f283c1017c0 2024-01-25T16:07:10.398 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:10.398 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:07:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 msgr2=0x7f28280a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 0x7f28280a38b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f283c100c60 tx=0x7f2838008040 comp rx=0 tx=0).stop 2024-01-25T16:07:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 msgr2=0x7f283c1a05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2834037040 tx=0x7f2834032a40 comp rx=0 tx=0).stop 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 shutdown_connections 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f28280a13f0 0x7f28280a38b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f283c1017c0 0x7f283c1a05e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f283c100e00 0x7f283c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 --2- 172.21.15.33:0/555642286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f283c0ffc10 0x7f283c199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 >> 172.21.15.33:0/555642286 conn(0x7f283c0fb180 msgr2=0x7f283c067ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 shutdown_connections 2024-01-25T16:07:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.397+0000 7f28441b0640 1 -- 172.21.15.33:0/555642286 wait complete. 2024-01-25T16:07:10.411 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:10.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- 172.21.15.33:0/2480477722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b81017c0 msgr2=0x7f80b8108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- 172.21.15.33:0/2480477722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b81017c0 0x7f80b8108050 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f80b0009ca0 tx=0x7f80b002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- 172.21.15.33:0/2480477722 shutdown_connections 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- 172.21.15.33:0/2480477722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b81017c0 0x7f80b8108050 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- 172.21.15.33:0/2480477722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b8100e00 0x7f80b8101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- 172.21.15.33:0/2480477722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b80ffc10 0x7f80b8100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- 172.21.15.33:0/2480477722 >> 172.21.15.33:0/2480477722 conn(0x7f80b80fb180 msgr2=0x7f80b80fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- 172.21.15.33:0/2480477722 shutdown_connections 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- 172.21.15.33:0/2480477722 wait complete. 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 Processor -- start 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- start start 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b8100e00 0x7f80b819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b81017c0 0x7f80b819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819e670 con 0x7f80b80ffc10 2024-01-25T16:07:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819e7e0 con 0x7f80b81017c0 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.513+0000 7f80c00da640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b819e9a0 con 0x7f80b8100e00 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40958/0 (socket says 172.21.15.33:40958) 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 -- 172.21.15.33:0/1612626115 learned_addr learned my addr 172.21.15.33:0/1612626115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bd675640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b8100e00 0x7f80b819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80be677640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b81017c0 0x7f80b819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 -- 172.21.15.33:0/1612626115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b8100e00 msgr2=0x7f80b819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b8100e00 0x7f80b819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 -- 172.21.15.33:0/1612626115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b81017c0 msgr2=0x7f80b819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b81017c0 0x7f80b819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 -- 172.21.15.33:0/1612626115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b0009900 con 0x7f80b80ffc10 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80bde76640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80b400c970 tx=0x7f80b400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4007b50 con 0x7f80b80ffc10 2024-01-25T16:07:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b4007cb0 con 0x7f80b80ffc10 2024-01-25T16:07:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b819ec20 con 0x7f80b80ffc10 2024-01-25T16:07:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400f740 con 0x7f80b80ffc10 2024-01-25T16:07:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.514+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b819f130 con 0x7f80b80ffc10 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.516+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b8106e60 con 0x7f80b80ffc10 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.517+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f80b400f8a0 con 0x7f80b80ffc10 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.517+0000 7f80aeffd640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 0x7f80a40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.517+0000 7f80bd675640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 0x7f80a40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.517+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f80b4013070 con 0x7f80b80ffc10 2024-01-25T16:07:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.518+0000 7f80bd675640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 0x7f80a40a3830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80b8068730 tx=0x7f80a800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.526+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f80b4096320 con 0x7f80b80ffc10 2024-01-25T16:07:10.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:10 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2572546171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:10 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4172626109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:10 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1281380908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:10 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2572546171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:10 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4172626109' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:10 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1281380908' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:10.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.707+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80b810b4b0 con 0x7f80b80ffc10 2024-01-25T16:07:10.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.710+0000 7f80aeffd640 1 -- 172.21.15.33:0/1612626115 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f80b4099940 con 0x7f80b80ffc10 2024-01-25T16:07:10.714 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:10.714 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 msgr2=0x7f80a40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 0x7f80a40a3830 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80b8068730 tx=0x7f80a800a2d0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 msgr2=0x7f80b80689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f80b400c970 tx=0x7f80b400cd70 comp rx=0 tx=0).stop 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 shutdown_connections 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80b81017c0 0x7f80b819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f80b8100e00 0x7f80b819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f80a40a1370 0x7f80a40a3830 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 --2- 172.21.15.33:0/1612626115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80b80ffc10 0x7f80b80689c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 >> 172.21.15.33:0/1612626115 conn(0x7f80b80fb180 msgr2=0x7f80b8104030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 shutdown_connections 2024-01-25T16:07:10.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.712+0000 7f80c00da640 1 -- 172.21.15.33:0/1612626115 wait complete. 2024-01-25T16:07:10.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:07:10.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 -- 172.21.15.33:0/1663317065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a8101770 msgr2=0x7fd2a8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 --2- 172.21.15.33:0/1663317065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a8101770 0x7fd2a8103b60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd2a4009900 tx=0x7fd2a402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 -- 172.21.15.33:0/1663317065 shutdown_connections 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 --2- 172.21.15.33:0/1663317065 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a81069d0 0x7fd2a8108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 --2- 172.21.15.33:0/1663317065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 0x7fd2a8106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 --2- 172.21.15.33:0/1663317065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a8101770 0x7fd2a8103b60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 -- 172.21.15.33:0/1663317065 >> 172.21.15.33:0/1663317065 conn(0x7fd2a80fb180 msgr2=0x7fd2a80fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:10.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 -- 172.21.15.33:0/1663317065 shutdown_connections 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.830+0000 7fd2b047c640 1 -- 172.21.15.33:0/1663317065 wait complete. 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 Processor -- start 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 -- start start 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a8101770 0x7fd2a8196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 0x7fd2a8197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819e800 con 0x7fd2a81069d0 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819e970 con 0x7fd2a81040a0 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2b047c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a819eb30 con 0x7fd2a8101770 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40986/0 (socket says 172.21.15.33:40986) 2024-01-25T16:07:10.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 -- 172.21.15.33:0/2696742972 learned_addr learned my addr 172.21.15.33:0/2696742972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2ada17640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 0x7fd2a8197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2ae218640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a8101770 0x7fd2a8196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 -- 172.21.15.33:0/2696742972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a8101770 msgr2=0x7fd2a8196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a8101770 0x7fd2a8196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 -- 172.21.15.33:0/2696742972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 msgr2=0x7fd2a8197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 0x7fd2a8197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.831+0000 7fd2aea19640 1 -- 172.21.15.33:0/2696742972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a4009560 con 0x7fd2a81069d0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd2aea19640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000eaa0 tx=0x7fd2a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000cdb0 con 0x7fd2a81069d0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd2a0005020 con 0x7fd2a81069d0 2024-01-25T16:07:10.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a819ee10 con 0x7fd2a81069d0 2024-01-25T16:07:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00116d0 con 0x7fd2a81069d0 2024-01-25T16:07:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.832+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a819f230 con 0x7fd2a81069d0 2024-01-25T16:07:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.833+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80fcc70 con 0x7fd2a81069d0 2024-01-25T16:07:10.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.835+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd2a0005190 con 0x7fd2a81069d0 2024-01-25T16:07:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.835+0000 7fd29f7fe640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 0x7fd2940a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.835+0000 7fd2ae218640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 0x7fd2940a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.835+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd2a00ccf40 con 0x7fd2a81069d0 2024-01-25T16:07:10.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.836+0000 7fd2ae218640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 0x7fd2940a3830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2a402c6b0 tx=0x7fd2a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:10.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:10.840+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd2a00961d0 con 0x7fd2a81069d0 2024-01-25T16:07:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.022+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2a810c7d0 con 0x7fd2a81069d0 2024-01-25T16:07:11.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.022+0000 7fd29f7fe640 1 -- 172.21.15.33:0/2696742972 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2a001f020 con 0x7fd2a81069d0 2024-01-25T16:07:11.026 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:07:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.024+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 msgr2=0x7fd2940a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.024+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 0x7fd2940a3830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2a402c6b0 tx=0x7fd2a4005fd0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.024+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 msgr2=0x7fd2a819e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.024+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000eaa0 tx=0x7fd2a000ef60 comp rx=0 tx=0).stop 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 shutdown_connections 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd2940a1370 0x7fd2940a3830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd2a81069d0 0x7fd2a819e290 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2a81040a0 0x7fd2a8197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 --2- 172.21.15.33:0/2696742972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd2a8101770 0x7fd2a8196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 >> 172.21.15.33:0/2696742972 conn(0x7fd2a80fb180 msgr2=0x7fd2a8102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 shutdown_connections 2024-01-25T16:07:11.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.025+0000 7fd2b047c640 1 -- 172.21.15.33:0/2696742972 wait complete. 2024-01-25T16:07:11.039 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:07:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 -- 172.21.15.33:0/2189424000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 msgr2=0x7f4204102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 --2- 172.21.15.33:0/2189424000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204102390 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009900 tx=0x7f41f402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 -- 172.21.15.33:0/2189424000 shutdown_connections 2024-01-25T16:07:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 --2- 172.21.15.33:0/2189424000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42041028d0 0x7f4204109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 --2- 172.21.15.33:0/2189424000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204102390 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 --2- 172.21.15.33:0/2189424000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4204100d20 0x7f4204101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 -- 172.21.15.33:0/2189424000 >> 172.21.15.33:0/2189424000 conn(0x7f42040fc290 msgr2=0x7f42040fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 -- 172.21.15.33:0/2189424000 shutdown_connections 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.141+0000 7f4208c9b640 1 -- 172.21.15.33:0/2189424000 wait complete. 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 Processor -- start 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 -- start start 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4204100d20 0x7f4204196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42041028d0 0x7f420419e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419e7d0 con 0x7f4204101f10 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419e940 con 0x7f4204100d20 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4208c9b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f420419eb00 con 0x7f42041028d0 2024-01-25T16:07:11.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4201d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4201d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41002/0 (socket says 172.21.15.33:41002) 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4201d9b640 1 -- 172.21.15.33:0/3500140400 learned_addr learned my addr 172.21.15.33:0/3500140400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f420259c640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4204100d20 0x7f4204196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.142+0000 7f4202d9d640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42041028d0 0x7f420419e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 -- 172.21.15.33:0/3500140400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42041028d0 msgr2=0x7f420419e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42041028d0 0x7f420419e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 -- 172.21.15.33:0/3500140400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4204100d20 msgr2=0x7f4204196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4204100d20 0x7f4204196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 -- 172.21.15.33:0/3500140400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4009560 con 0x7f4204101f10 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4201d9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009230 tx=0x7f41f4004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f40043d0 con 0x7f4204101f10 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420419ed80 con 0x7f4204101f10 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f41f40054c0 con 0x7f4204101f10 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f403e450 con 0x7f4204101f10 2024-01-25T16:07:11.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.143+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420419f230 con 0x7f4204101f10 2024-01-25T16:07:11.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.144+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42041080c0 con 0x7f4204101f10 2024-01-25T16:07:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.145+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f41f40035f0 con 0x7f4204101f10 2024-01-25T16:07:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.146+0000 7f41f37fe640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 0x7f41ec0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.146+0000 7f420259c640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 0x7f41ec0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.146+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f41f40ef920 con 0x7f4204101f10 2024-01-25T16:07:11.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.147+0000 7f420259c640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 0x7f41ec0a3830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4204197b90 tx=0x7f41f800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.150+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f41f40b8d10 con 0x7f4204101f10 2024-01-25T16:07:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:11 smithi033 ceph-mon[31417]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:11 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/555642286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:11 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1612626115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:11 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2696742972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.329+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f420402cb40 con 0x7f4204101f10 2024-01-25T16:07:11.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.330+0000 7f41f37fe640 1 -- 172.21.15.33:0/3500140400 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41f40bc330 con 0x7f4204101f10 2024-01-25T16:07:11.333 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:07:11.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 msgr2=0x7f41ec0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 0x7f41ec0a3830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4204197b90 tx=0x7f41f800b390 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 msgr2=0x7f4204197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009230 tx=0x7f41f4004ee0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 shutdown_connections 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42041028d0 0x7f420419e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f41ec0a1370 0x7f41ec0a3830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4204101f10 0x7f4204197200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 --2- 172.21.15.33:0/3500140400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4204100d20 0x7f4204196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 >> 172.21.15.33:0/3500140400 conn(0x7f42040fc290 msgr2=0x7f4204105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 shutdown_connections 2024-01-25T16:07:11.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.332+0000 7f4208c9b640 1 -- 172.21.15.33:0/3500140400 wait complete. 2024-01-25T16:07:11.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:07:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:11 smithi076 ceph-mon[35642]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:11 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/555642286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:11 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1612626115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:11 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2696742972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.447+0000 7f8c71be5640 1 -- 172.21.15.33:0/2109401160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c101770 msgr2=0x7f8c6c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.447+0000 7f8c71be5640 1 --2- 172.21.15.33:0/2109401160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c101770 0x7f8c6c103b60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009900 tx=0x7f8c5c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- 172.21.15.33:0/2109401160 shutdown_connections 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- 172.21.15.33:0/2109401160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c108e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- 172.21.15.33:0/2109401160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c106490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- 172.21.15.33:0/2109401160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c101770 0x7f8c6c103b60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- 172.21.15.33:0/2109401160 >> 172.21.15.33:0/2109401160 conn(0x7f8c6c0fb180 msgr2=0x7f8c6c0fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- 172.21.15.33:0/2109401160 shutdown_connections 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- 172.21.15.33:0/2109401160 wait complete. 2024-01-25T16:07:11.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 Processor -- start 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- start start 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 0x7f8c6c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c19e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19e800 con 0x7f8c6c1040a0 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19e970 con 0x7f8c6c101770 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.448+0000 7f8c71be5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c19eb30 con 0x7f8c6c1069d0 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41016/0 (socket says 172.21.15.33:41016) 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 -- 172.21.15.33:0/4173396033 learned_addr learned my addr 172.21.15.33:0/4173396033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6bfff640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c19e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6b7fe640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 0x7f8c6c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 -- 172.21.15.33:0/4173396033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 msgr2=0x7f8c6c19e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c19e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 -- 172.21.15.33:0/4173396033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 msgr2=0x7f8c6c196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 0x7f8c6c196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 -- 172.21.15.33:0/4173396033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009560 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6b7fe640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 0x7f8c6c196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6bfff640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c6affd640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c970 tx=0x7f8c6000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60004440 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c19ee10 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c60007d20 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000f450 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.449+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c19f2b0 con 0x7f8c6c1040a0 2024-01-25T16:07:11.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.450+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c0fcc70 con 0x7f8c6c1040a0 2024-01-25T16:07:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.451+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f8c600035f0 con 0x7f8c6c1040a0 2024-01-25T16:07:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.452+0000 7f8c68ff9640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 0x7f8c540a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.452+0000 7f8c6b7fe640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 0x7f8c540a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.452+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8c600ccfe0 con 0x7f8c6c1040a0 2024-01-25T16:07:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.452+0000 7f8c6b7fe640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 0x7f8c540a3830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036040 tx=0x7f8c5c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.456+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8c60096230 con 0x7f8c6c1040a0 2024-01-25T16:07:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:11 smithi152 ceph-mon[36015]: pgmap v329: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:11 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/555642286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:11 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1612626115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:11 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2696742972' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.634+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c6c10c820 con 0x7f8c6c1040a0 2024-01-25T16:07:11.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.635+0000 7f8c68ff9640 1 -- 172.21.15.33:0/4173396033 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c600140a0 con 0x7f8c6c1040a0 2024-01-25T16:07:11.638 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:07:11.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 msgr2=0x7f8c540a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 0x7f8c540a3830 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036040 tx=0x7f8c5c000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 msgr2=0x7f8c6c197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c970 tx=0x7f8c6000ce30 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 shutdown_connections 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8c6c1069d0 0x7f8c6c19e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f8c540a1370 0x7f8c540a3830 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c6c1040a0 0x7f8c6c197260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 --2- 172.21.15.33:0/4173396033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c6c101770 0x7f8c6c196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 >> 172.21.15.33:0/4173396033 conn(0x7f8c6c0fb180 msgr2=0x7f8c6c1020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 shutdown_connections 2024-01-25T16:07:11.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.637+0000 7f8c71be5640 1 -- 172.21.15.33:0/4173396033 wait complete. 2024-01-25T16:07:11.652 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 -- 172.21.15.33:0/205948112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 msgr2=0x7f6004101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 --2- 172.21.15.33:0/205948112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004101130 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009900 tx=0x7f5ffc02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 -- 172.21.15.33:0/205948112 shutdown_connections 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 --2- 172.21.15.33:0/205948112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f6004109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 --2- 172.21.15.33:0/205948112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 0x7f6004102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 --2- 172.21.15.33:0/205948112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004101130 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 -- 172.21.15.33:0/205948112 >> 172.21.15.33:0/205948112 conn(0x7f60040fc2a0 msgr2=0x7f60040fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 -- 172.21.15.33:0/205948112 shutdown_connections 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 -- 172.21.15.33:0/205948112 wait complete. 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.750+0000 7f6008877640 1 Processor -- start 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 -- start start 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 0x7f6004197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f600419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419e6e0 con 0x7f6004100d10 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419e850 con 0x7f60041028c0 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600419ea10 con 0x7f6004101f00 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41038/0 (socket says 172.21.15.33:41038) 2024-01-25T16:07:11.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 -- 172.21.15.33:0/1552138679 learned_addr learned my addr 172.21.15.33:0/1552138679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6002d9d640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f600419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6001d9b640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 0x7f6004197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 -- 172.21.15.33:0/1552138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 msgr2=0x7f6004197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 0x7f6004197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 -- 172.21.15.33:0/1552138679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 msgr2=0x7f600419e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f600419e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 -- 172.21.15.33:0/1552138679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc009560 con 0x7f6004100d10 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6002d9d640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f600419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f600259c640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009230 tx=0x7f5ffc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0044c0 con 0x7f6004100d10 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ffc0054c0 con 0x7f6004100d10 2024-01-25T16:07:11.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600419ec90 con 0x7f6004100d10 2024-01-25T16:07:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.751+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc03e450 con 0x7f6004100d10 2024-01-25T16:07:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.752+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600419f1a0 con 0x7f6004100d10 2024-01-25T16:07:11.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.752+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004107fd0 con 0x7f6004100d10 2024-01-25T16:07:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.754+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5ffc0036f0 con 0x7f6004100d10 2024-01-25T16:07:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.754+0000 7f5ff37fe640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 0x7f5fec0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.754+0000 7f6001d9b640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 0x7f5fec0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.754+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5ffc0efb20 con 0x7f6004100d10 2024-01-25T16:07:11.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.755+0000 7f6001d9b640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 0x7f5fec0a3830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6004198100 tx=0x7f5ff8005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:11.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.758+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5ffc0b8f50 con 0x7f6004100d10 2024-01-25T16:07:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.974+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f600410c760 con 0x7f6004100d10 2024-01-25T16:07:11.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.975+0000 7f5ff37fe640 1 -- 172.21.15.33:0/1552138679 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f5ffc0bc570 con 0x7f6004100d10 2024-01-25T16:07:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:11.979 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:07:11.979 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:11.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 msgr2=0x7f5fec0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 0x7f5fec0a3830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6004198100 tx=0x7f5ff8005cc0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 msgr2=0x7f6004196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc009230 tx=0x7f5ffc0040c0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 shutdown_connections 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60041028c0 0x7f600419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6004101f00 0x7f6004197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5fec0a1370 0x7f5fec0a3830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 --2- 172.21.15.33:0/1552138679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6004100d10 0x7f6004196bf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 >> 172.21.15.33:0/1552138679 conn(0x7f60040fc2a0 msgr2=0x7f60041051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 shutdown_connections 2024-01-25T16:07:11.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:11.977+0000 7f6008877640 1 -- 172.21.15.33:0/1552138679 wait complete. 2024-01-25T16:07:11.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- 172.21.15.33:0/2698517988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc100e00 msgr2=0x7f6ddc101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 --2- 172.21.15.33:0/2698517988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc100e00 0x7f6ddc101280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc009900 tx=0x7f6dcc02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- 172.21.15.33:0/2698517988 shutdown_connections 2024-01-25T16:07:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 --2- 172.21.15.33:0/2698517988 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 --2- 172.21.15.33:0/2698517988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc100e00 0x7f6ddc101280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 --2- 172.21.15.33:0/2698517988 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 0x7f6ddc100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- 172.21.15.33:0/2698517988 >> 172.21.15.33:0/2698517988 conn(0x7f6ddc0fb1a0 msgr2=0x7f6ddc0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- 172.21.15.33:0/2698517988 shutdown_connections 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- 172.21.15.33:0/2698517988 wait complete. 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 Processor -- start 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 -- start start 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.091+0000 7f6de416c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 0x7f6ddc196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc100e00 0x7f6ddc197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19e7d0 con 0x7f6ddc1017c0 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19e940 con 0x7f6ddc0ffc10 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc19eb00 con 0x7f6ddc100e00 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41058/0 (socket says 172.21.15.33:41058) 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 -- 172.21.15.33:0/2707165488 learned_addr learned my addr 172.21.15.33:0/2707165488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de1f08640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 0x7f6ddc196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de1707640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc100e00 0x7f6ddc197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 -- 172.21.15.33:0/2707165488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc100e00 msgr2=0x7f6ddc197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc100e00 0x7f6ddc197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 -- 172.21.15.33:0/2707165488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 msgr2=0x7f6ddc196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 0x7f6ddc196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 -- 172.21.15.33:0/2707165488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc009560 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de2709640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cad0 tx=0x7f6dd400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400ae20 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6dd4005020 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc19ede0 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd400f6b0 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.092+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc19f260 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.093+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc106fb0 con 0x7f6ddc1017c0 2024-01-25T16:07:12.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.094+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f6dd4005190 con 0x7f6ddc1017c0 2024-01-25T16:07:12.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.095+0000 7f6dd2ffd640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 0x7f6dc80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.095+0000 7f6de1f08640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 0x7f6dc80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.095+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6dd40cd1d0 con 0x7f6ddc1017c0 2024-01-25T16:07:12.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.095+0000 7f6de1f08640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 0x7f6dc80a38b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005db0 tx=0x7f6dd8005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.099+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6dd4096420 con 0x7f6ddc1017c0 2024-01-25T16:07:12.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.279+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ddc10c820 con 0x7f6ddc1017c0 2024-01-25T16:07:12.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.280+0000 7f6dd2ffd640 1 -- 172.21.15.33:0/2707165488 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f6dd401d090 con 0x7f6ddc1017c0 2024-01-25T16:07:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:12.284 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 msgr2=0x7f6dc80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 0x7f6dc80a38b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8005db0 tx=0x7f6dd8005d40 comp rx=0 tx=0).stop 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 msgr2=0x7f6ddc19e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6dd400cad0 tx=0x7f6dd400cf90 comp rx=0 tx=0).stop 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 shutdown_connections 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6dc80a13f0 0x7f6dc80a38b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ddc1017c0 0x7f6ddc19e260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ddc100e00 0x7f6ddc197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 --2- 172.21.15.33:0/2707165488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ddc0ffc10 0x7f6ddc196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 >> 172.21.15.33:0/2707165488 conn(0x7f6ddc0fb1a0 msgr2=0x7f6ddc104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 shutdown_connections 2024-01-25T16:07:12.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.282+0000 7f6de416c640 1 -- 172.21.15.33:0/2707165488 wait complete. 2024-01-25T16:07:12.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3500140400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4173396033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.295 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:12 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1552138679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:12.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 -- 172.21.15.33:0/396068486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78840ffc30 msgr2=0x7f7884100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 --2- 172.21.15.33:0/396068486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78840ffc30 0x7f7884100050 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 -- 172.21.15.33:0/396068486 shutdown_connections 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 --2- 172.21.15.33:0/396068486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78841017e0 0x7f7884108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 --2- 172.21.15.33:0/396068486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 0x7f78841012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 --2- 172.21.15.33:0/396068486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78840ffc30 0x7f7884100050 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 -- 172.21.15.33:0/396068486 >> 172.21.15.33:0/396068486 conn(0x7f78840fb1a0 msgr2=0x7f78840fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 -- 172.21.15.33:0/396068486 shutdown_connections 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.395+0000 7f788b943640 1 -- 172.21.15.33:0/396068486 wait complete. 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 Processor -- start 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 -- start start 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78840ffc30 0x7f7884069e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 0x7f788419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78841a0a90 con 0x7f78841017e0 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78841a0c00 con 0x7f78840ffc30 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f788b943640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78841a0dc0 con 0x7f7884100e20 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41082/0 (socket says 172.21.15.33:41082) 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 -- 172.21.15.33:0/3144170252 learned_addr learned my addr 172.21.15.33:0/3144170252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f78896df640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78840ffc30 0x7f7884069e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7888ede640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 0x7f788419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 -- 172.21.15.33:0/3144170252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 msgr2=0x7f788419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 0x7f788419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 -- 172.21.15.33:0/3144170252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78840ffc30 msgr2=0x7f7884069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78840ffc30 0x7f7884069e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 -- 172.21.15.33:0/3144170252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009560 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f7889ee0640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f787c00cad0 tx=0x7f787c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.396+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00ae20 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.397+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78841a0fe0 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.397+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f787c005020 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.397+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00f6b0 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.397+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78841a14c0 con 0x7f78841017e0 2024-01-25T16:07:12.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.397+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7884107070 con 0x7f78841017e0 2024-01-25T16:07:12.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.399+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f787c005190 con 0x7f78841017e0 2024-01-25T16:07:12.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.399+0000 7f787a7fc640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 0x7f78700a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.399+0000 7f78896df640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 0x7f78700a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.400+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f787c0cd040 con 0x7f78841017e0 2024-01-25T16:07:12.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.400+0000 7f78896df640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 0x7f78700a3860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f788002c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.403+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f787c096450 con 0x7f78841017e0 2024-01-25T16:07:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.619+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f788410b6e0 con 0x7f78841017e0 2024-01-25T16:07:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.620+0000 7f787a7fc640 1 -- 172.21.15.33:0/3144170252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f787c01d020 con 0x7f78841017e0 2024-01-25T16:07:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:12.624 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:07:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 msgr2=0x7f78700a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 0x7f78700a3860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f788002c5d0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 msgr2=0x7f78841a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f787c00cad0 tx=0x7f787c00cf90 comp rx=0 tx=0).stop 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 shutdown_connections 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f78700a13a0 0x7f78700a3860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78841017e0 0x7f78841a03d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7884100e20 0x7f788419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 --2- 172.21.15.33:0/3144170252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78840ffc30 0x7f7884069e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 >> 172.21.15.33:0/3144170252 conn(0x7f78840fb1a0 msgr2=0x7f7884104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 shutdown_connections 2024-01-25T16:07:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.622+0000 7f788b943640 1 -- 172.21.15.33:0/3144170252 wait complete. 2024-01-25T16:07:12.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:12 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3500140400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:12 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4173396033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:12 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1552138679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:12.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-25T16:05:46.107985+0000', 'modified': '2024-01-25T16:05:50.161531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24517, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6835/3469230010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3469230010}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3469230010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/1042830191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 1042830191}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 1042830191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/2179148160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 2179148160}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 2179148160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.76:6837/2322550881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2322550881}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2322550881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.152:6839/2298219777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 2298219777}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 2298219777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6839/406718297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 406718297}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 406718297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-25T16:07:12.637 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-25T16:07:12.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-01-25T16:07:12.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:12 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3500140400' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:12 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4173396033' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:12.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:12 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1552138679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 -- 172.21.15.33:0/4014474024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec101ef0 msgr2=0x7f39ec102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/4014474024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec101ef0 0x7f39ec102370 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009900 tx=0x7f39e002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 -- 172.21.15.33:0/4014474024 shutdown_connections 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/4014474024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec1028b0 0x7f39ec109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/4014474024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec101ef0 0x7f39ec102370 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/4014474024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec100d00 0x7f39ec101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 -- 172.21.15.33:0/4014474024 >> 172.21.15.33:0/4014474024 conn(0x7f39ec0fc290 msgr2=0x7f39ec0fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 -- 172.21.15.33:0/4014474024 shutdown_connections 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.737+0000 7f39f0a5d640 1 -- 172.21.15.33:0/4014474024 wait complete. 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 Processor -- start 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 -- start start 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 0x7f39ec196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec101ef0 0x7f39ec197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19e7e0 con 0x7f39ec1028b0 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19e950 con 0x7f39ec101ef0 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39f0a5d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ec19eb10 con 0x7f39ec100d00 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39ead9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41090/0 (socket says 172.21.15.33:41090) 2024-01-25T16:07:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39ead9d640 1 -- 172.21.15.33:0/2702450112 learned_addr learned my addr 172.21.15.33:0/2702450112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec101ef0 0x7f39ec197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.738+0000 7f39ea59c640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 0x7f39ec196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 -- 172.21.15.33:0/2702450112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 msgr2=0x7f39ec196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 0x7f39ec196d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 -- 172.21.15.33:0/2702450112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec101ef0 msgr2=0x7f39ec197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec101ef0 0x7f39ec197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 -- 172.21.15.33:0/2702450112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e0009560 con 0x7f39ec1028b0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ea59c640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 0x7f39ec196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39ead9d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00cad0 tx=0x7f39dc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39db7fe640 1 -- 172.21.15.33:0/2702450112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00ae20 con 0x7f39ec1028b0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec19edf0 con 0x7f39ec1028b0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39db7fe640 1 -- 172.21.15.33:0/2702450112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39dc005020 con 0x7f39ec1028b0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39db7fe640 1 -- 172.21.15.33:0/2702450112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc00f6b0 con 0x7f39ec1028b0 2024-01-25T16:07:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.739+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec19f1c0 con 0x7f39ec1028b0 2024-01-25T16:07:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.741+0000 7f39db7fe640 1 -- 172.21.15.33:0/2702450112 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f39dc005190 con 0x7f39ec1028b0 2024-01-25T16:07:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.741+0000 7f39f3618740 1 Processor -- start 2024-01-25T16:07:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 -- start start 2024-01-25T16:07:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c6798ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 0x5592c679db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c6707030 con 0x5592c679e200 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c6704720 con 0x5592c6799090 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39f3618740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c6700010 con 0x5592c67986a0 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41096/0 (socket says 172.21.15.33:41096) 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 -- 172.21.15.33:0/980516542 learned_addr learned my addr 172.21.15.33:0/980516542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c6798ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39db7fe640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 0x7f39d40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39ea59c640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 0x5592c679db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39ead9d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 0x7f39d40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 -- 172.21.15.33:0/980516542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 msgr2=0x5592c6798ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c6798ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 -- 172.21.15.33:0/980516542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 msgr2=0x5592c679db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 0x5592c679db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 -- 172.21.15.33:0/980516542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c66c2680 con 0x5592c679e200 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39ea59c640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 0x5592c679db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39db7fe640 1 -- 172.21.15.33:0/2702450112 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39dc0cce30 con 0x7f39ec1028b0 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.742+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f39e0036040 tx=0x7f39e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99eb6226f146bdb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39d8ff9640 1 -- 172.21.15.33:0/980516542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e0004a10 con 0x5592c679e200 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39ead9d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 0x7f39d40a3830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f39dc099b10 tx=0x7f39dc06cee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39d8ff9640 1 -- 172.21.15.33:0/980516542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e0004500 con 0x5592c679e200 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39d8ff9640 1 -- 172.21.15.33:0/980516542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e00035f0 con 0x5592c679e200 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 -- 172.21.15.33:0/980516542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 msgr2=0x5592c67a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f39e0036040 tx=0x7f39e0005fd0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 -- 172.21.15.33:0/980516542 shutdown_connections 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c679e200 0x5592c67a0690 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6799090 0x5592c679db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 --2- 172.21.15.33:0/980516542 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c6798ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 -- 172.21.15.33:0/980516542 >> 172.21.15.33:0/980516542 conn(0x5592c6793c10 msgr2=0x5592c6794010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 -- 172.21.15.33:0/980516542 shutdown_connections 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.743+0000 7f39f3618740 1 -- 172.21.15.33:0/980516542 wait complete. 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 -- start start 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 Processor -- start 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c679e200 0x5592c67a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c66b95f0 con 0x5592c6799090 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c66b9290 con 0x5592c679e200 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c667e300 con 0x5592c67986a0 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39f3618740 10 client.0 _ll_register_callbacks cb 0x5592c66b4260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:12.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41106/0 (socket says 172.21.15.33:41106) 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 -- 172.21.15.33:0/2318029525 learned_addr learned my addr 172.21.15.33:0/2318029525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c679e200 0x5592c67a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 -- 172.21.15.33:0/2318029525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 msgr2=0x5592c67a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 -- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c679e200 msgr2=0x5592c67a9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c679e200 0x5592c67a9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39ea59c640 1 -- 172.21.15.33:0/2318029525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ec109a40 con 0x5592c6799090 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.744+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39ea59c640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39e4010810 tx=0x7f39e4010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4461239379f07e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4004440 con 0x5592c6799090 2024-01-25T16:07:12.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39e4017030 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c66f8bc0 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e4010ec0 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 10 client.15066 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5592c66fe690 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592c66f8bc0 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39e40047a0 con 0x5592c6799090 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 10 client.15066 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 20 client.15066 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c67afb00 0x5592c67ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 4 client.15066 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.745+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 --> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592c63ff6c0 con 0x5592c67afb00 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.746+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c67afb00 0x5592c67ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.746+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c67afb00 0x5592c67ae510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=462f9dc92782c5b1 server_cookie=5970af56044d1333 in_seq=0 out_seq=0 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.746+0000 7f39d1ffb640 10 client.15066 ms_handle_connect on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39d1ffb640 1 -- 172.21.15.33:0/2318029525 <== mds.2 v2:172.21.15.76:6836/2322550881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592c63ff6c0 con 0x5592c67afb00 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39d1ffb640 10 client.15066 handle_command_reply: tid=0 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 client.15066 shutdown 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 msgr2=0x5592c67a2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f39e4010810 tx=0x7f39e4010cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 shutdown_connections 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c67afb00 0x5592c67ae510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c679e200 0x5592c67a9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39d1ffb640 0 client.15066 ms_handle_reset on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a2e80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 --2- 172.21.15.33:0/2318029525 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 >> 172.21.15.33:0/2318029525 conn(0x5592c6793c10 msgr2=0x5592c6792da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 shutdown_connections 2024-01-25T16:07:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 1 -- 172.21.15.33:0/2318029525 wait complete. 2024-01-25T16:07:12.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.747+0000 7f39f3618740 20 client.15066 trim_cache size 0 max 16384 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.762+0000 7f39f3618740 1 Processor -- start 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.762+0000 7f39f3618740 1 -- start start 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c6794c00 con 0x5592c6799090 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c63ff6c0 con 0x5592c6793c10 2024-01-25T16:07:12.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39f3618740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c66f8bc0 con 0x5592c67986a0 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41116/0 (socket says 172.21.15.33:41116) 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 -- 172.21.15.33:0/1891688283 learned_addr learned my addr 172.21.15.33:0/1891688283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39ea59c640 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 -- 172.21.15.33:0/1891688283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 msgr2=0x5592c67a2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 -- 172.21.15.33:0/1891688283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 msgr2=0x5592c67a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 -- 172.21.15.33:0/1891688283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c669b390 con 0x5592c6799090 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39ea59c640 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f39e00058a0 tx=0x7f39e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fd61fb0e65517 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39d3fff640 1 -- 172.21.15.33:0/1891688283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e002cb30 con 0x5592c6799090 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39d3fff640 1 -- 172.21.15.33:0/1891688283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39e002c950 con 0x5592c6799090 2024-01-25T16:07:12.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.763+0000 7f39d3fff640 1 -- 172.21.15.33:0/1891688283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e00585b0 con 0x5592c6799090 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- 172.21.15.33:0/1891688283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 msgr2=0x5592c67a9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f39e00058a0 tx=0x7f39e0005fd0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- 172.21.15.33:0/1891688283 shutdown_connections 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c6799090 0x5592c67a9e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c67986a0 0x5592c67a2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- 172.21.15.33:0/1891688283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- 172.21.15.33:0/1891688283 >> 172.21.15.33:0/1891688283 conn(0x5592c6794360 msgr2=0x5592c679a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- 172.21.15.33:0/1891688283 shutdown_connections 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- 172.21.15.33:0/1891688283 wait complete. 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- start start 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 Processor -- start 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c6799090 0x5592c67a6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c67a7090 con 0x5592c67986a0 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c67a9fe0 con 0x5592c6793c10 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c67aa1a0 con 0x5592c6799090 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39f3618740 10 client.0 _ll_register_callbacks cb 0x7f39e002c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.764+0000 7f39ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41132/0 (socket says 172.21.15.33:41132) 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 -- 172.21.15.33:0/75488916 learned_addr learned my addr 172.21.15.33:0/75488916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 -- 172.21.15.33:0/75488916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c6799090 msgr2=0x5592c67a6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c6799090 0x5592c67a6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 -- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 msgr2=0x5592c67a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 -- 172.21.15.33:0/75488916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592c66c2680 con 0x5592c67986a0 2024-01-25T16:07:12.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39ea59c640 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39e400d010 tx=0x7f39e4010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e95b56a93255b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4004880 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592c66f8ea0 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f39e4000e40 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39e4004450 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39f3618740 10 client.15072 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5592c66c0ee0 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5592c66f8ea0 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.765+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39e40038f0 con 0x5592c67986a0 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 10 client.15072 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 20 client.15072 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c679e200 0x5592c6805fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 4 client.15072 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 --> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5592c6452d10 con 0x5592c679e200 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c679e200 0x5592c6805fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39d1ffb640 10 client.15072 ms_handle_connect on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39e9d9b640 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c679e200 0x5592c6805fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=963408700b5e25fd server_cookie=a1d09cf706a27792 in_seq=0 out_seq=0 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39d1ffb640 1 -- 172.21.15.33:0/75488916 <== mds.2 v2:172.21.15.76:6836/2322550881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5592c6452d10 con 0x5592c679e200 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39d1ffb640 10 client.15072 handle_command_reply: tid=0 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 1 client.15072 shutdown 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 msgr2=0x5592c67a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.766+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f39e400d010 tx=0x7f39e4010f50 comp rx=0 tx=0).stop 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 shutdown_connections 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x5592c679e200 0x5592c6805fd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5592c6799090 0x5592c67a6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39d1ffb640 0 client.15072 ms_handle_reset on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:12.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592c67986a0 0x5592c67a6450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 --2- 172.21.15.33:0/75488916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592c6793c10 0x5592c67a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 >> 172.21.15.33:0/75488916 conn(0x5592c6794360 msgr2=0x5592c679a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 shutdown_connections 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 1 -- 172.21.15.33:0/75488916 wait complete. 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f3618740 20 client.15072 trim_cache size 0 max 16384 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24494, 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:12.778 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 82.958984783000005, 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 102.363156008 2024-01-25T16:07:12.779 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 msgr2=0x7f39d40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 0x7f39d40a3830 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f39dc099b10 tx=0x7f39dc06cee0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 msgr2=0x7f39ec19e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00cad0 tx=0x7f39dc00cf90 comp rx=0 tx=0).stop 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 shutdown_connections 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f39d40a1370 0x7f39d40a3830 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39ec1028b0 0x7f39ec19e270 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39ec101ef0 0x7f39ec197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 --2- 172.21.15.33:0/2702450112 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f39ec100d00 0x7f39ec196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 >> 172.21.15.33:0/2702450112 conn(0x7f39ec0fc290 msgr2=0x7f39ec105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 shutdown_connections 2024-01-25T16:07:12.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.767+0000 7f39f0a5d640 1 -- 172.21.15.33:0/2702450112 wait complete. 2024-01-25T16:07:12.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-25T16:07:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 -- 172.21.15.33:0/3207978854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730101fa0 msgr2=0x7f6730102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 --2- 172.21.15.33:0/3207978854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730101fa0 0x7f6730102420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f671c009900 tx=0x7f671c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 -- 172.21.15.33:0/3207978854 shutdown_connections 2024-01-25T16:07:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 --2- 172.21.15.33:0/3207978854 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730102960 0x7f67301091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 --2- 172.21.15.33:0/3207978854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730101fa0 0x7f6730102420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 --2- 172.21.15.33:0/3207978854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 0x7f67300ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 -- 172.21.15.33:0/3207978854 >> 172.21.15.33:0/3207978854 conn(0x7f67300fa900 msgr2=0x7f67300fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 -- 172.21.15.33:0/3207978854 shutdown_connections 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.881+0000 7f6736dec640 1 -- 172.21.15.33:0/3207978854 wait complete. 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 Processor -- start 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 -- start start 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 0x7f6730196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730101fa0 0x7f6730197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019e7d0 con 0x7f6730102960 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019e940 con 0x7f67300fefd0 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6736dec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019eb00 con 0x7f6730101fa0 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41148/0 (socket says 172.21.15.33:41148) 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 -- 172.21.15.33:0/2132556382 learned_addr learned my addr 172.21.15.33:0/2132556382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f672ffff640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730101fa0 0x7f6730197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6734b88640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 0x7f6730196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 -- 172.21.15.33:0/2132556382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730101fa0 msgr2=0x7f6730197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730101fa0 0x7f6730197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 -- 172.21.15.33:0/2132556382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 msgr2=0x7f6730196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 0x7f6730196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 -- 172.21.15.33:0/2132556382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c009560 con 0x7f6730102960 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.882+0000 7f6735389640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f672800eaa0 tx=0x7f672800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.883+0000 7f672dffb640 1 -- 172.21.15.33:0/2132556382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800cdb0 con 0x7f6730102960 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.883+0000 7f672dffb640 1 -- 172.21.15.33:0/2132556382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6728005020 con 0x7f6730102960 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.883+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673019ede0 con 0x7f6730102960 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.883+0000 7f672dffb640 1 -- 172.21.15.33:0/2132556382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280116d0 con 0x7f6730102960 2024-01-25T16:07:12.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.883+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673019f260 con 0x7f6730102960 2024-01-25T16:07:12.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f672dffb640 1 -- 172.21.15.33:0/2132556382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f6728005190 con 0x7f6730102960 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 Processor -- start 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 -- start start 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6ba9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f672dffb640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 0x7f67200a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bb1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6b18030 con 0x5650c6baa090 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6b15720 con 0x5650c6ba96a0 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.885+0000 7f67399b2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6b11010 con 0x5650c6baf200 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f672ffff640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 0x7f67200a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41160/0 (socket says 172.21.15.33:41160) 2024-01-25T16:07:12.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 -- 172.21.15.33:0/3115326551 learned_addr learned my addr 172.21.15.33:0/3115326551 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f672dffb640 1 -- 172.21.15.33:0/2132556382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67280ccef0 con 0x7f6730102960 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6735389640 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bb1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f672ffff640 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6ba9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 -- 172.21.15.33:0/3115326551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 msgr2=0x5650c6bb1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bb1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 -- 172.21.15.33:0/3115326551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 msgr2=0x5650c6ba9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6ba9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 -- 172.21.15.33:0/3115326551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c6ad3680 con 0x5650c6baa090 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f672ffff640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 0x7f67200a3830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f671c005ab0 tx=0x7f671c02cbf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f6734b88640 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6724009b40 tx=0x7f672400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125023c8cd7a878a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f671affd640 1 -- 172.21.15.33:0/3115326551 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400ed20 con 0x5650c6baa090 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f671affd640 1 -- 172.21.15.33:0/3115326551 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672400ee80 con 0x5650c6baa090 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.886+0000 7f671affd640 1 -- 172.21.15.33:0/3115326551 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240107f0 con 0x5650c6baa090 2024-01-25T16:07:12.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- 172.21.15.33:0/3115326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 msgr2=0x5650c6baeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6724009b40 tx=0x7f672400e2f0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- 172.21.15.33:0/3115326551 shutdown_connections 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bb1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6baeb90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 --2- 172.21.15.33:0/3115326551 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6ba9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- 172.21.15.33:0/3115326551 >> 172.21.15.33:0/3115326551 conn(0x5650c6ba4c10 msgr2=0x5650c6ba5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- 172.21.15.33:0/3115326551 shutdown_connections 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- 172.21.15.33:0/3115326551 wait complete. 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 -- start start 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.887+0000 7f67399b2740 1 Processor -- start 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baa090 0x5650c6bb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baf200 0x5650c6bbad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6aca5f0 con 0x5650c6ba96a0 2024-01-25T16:07:12.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6aca290 con 0x5650c6baf200 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6a8f300 con 0x5650c6baa090 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 10 client.0 _ll_register_callbacks cb 0x5650c6ac5260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41162/0 (socket says 172.21.15.33:41162) 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 -- 172.21.15.33:0/4293760567 learned_addr learned my addr 172.21.15.33:0/4293760567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f672ffff640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baa090 0x5650c6bb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 -- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baa090 msgr2=0x5650c6bb3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baa090 0x5650c6bb3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 -- 172.21.15.33:0/4293760567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baf200 msgr2=0x5650c6bbad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baf200 0x5650c6bbad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 -- 172.21.15.33:0/4293760567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6724009770 con 0x5650c6ba96a0 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f6734b88640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f672400e7f0 tx=0x7f67240053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d73b442f8b22d4b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f670ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724020070 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6724011470 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.888+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650c6b09ea0 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6724011630 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 10 client.15084 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5650c6b0f690 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650c6b09ea0 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f672400c070 con 0x5650c6ba96a0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 10 client.15084 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 20 client.15084 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6bc08c0 0x5650c6bbf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 4 client.15084 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 --> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650c6b09bc0 con 0x5650c6bc08c0 2024-01-25T16:07:12.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.889+0000 7f672ffff640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6bc08c0 0x5650c6bbf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.890+0000 7f670ffff640 10 client.15084 ms_handle_connect on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.890+0000 7f672ffff640 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6bc08c0 0x5650c6bbf2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac3e27441811e386 server_cookie=c63b0ee205b34f27 in_seq=0 out_seq=0 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.890+0000 7f670ffff640 1 -- 172.21.15.33:0/4293760567 <== mds.1 v2:172.21.15.152:6838/2298219777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650c6b09bc0 con 0x5650c6bc08c0 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.890+0000 7f670ffff640 10 client.15084 handle_command_reply: tid=0 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.890+0000 7f67399b2740 1 client.15084 shutdown 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 msgr2=0x5650c6bb37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f672400e7f0 tx=0x7f67240053b0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 shutdown_connections 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6bc08c0 0x5650c6bbf2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baf200 0x5650c6bbad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f670ffff640 0 client.15084 ms_handle_reset on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baa090 0x5650c6bb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 --2- 172.21.15.33:0/4293760567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6ba96a0 0x5650c6bb37d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 >> 172.21.15.33:0/4293760567 conn(0x5650c6ba4c10 msgr2=0x5650c6ba3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 shutdown_connections 2024-01-25T16:07:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 1 -- 172.21.15.33:0/4293760567 wait complete. 2024-01-25T16:07:12.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.891+0000 7f67399b2740 20 client.15084 trim_cache size 0 max 16384 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.906+0000 7f67399b2740 1 Processor -- start 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.906+0000 7f67399b2740 1 -- start start 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6bbf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bbf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6baadf0 con 0x5650c6baa090 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6b09bc0 con 0x5650c6ba96a0 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f67399b2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6b09ea0 con 0x5650c6baf200 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41166/0 (socket says 172.21.15.33:41166) 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 -- 172.21.15.33:0/2887732027 learned_addr learned my addr 172.21.15.33:0/2887732027 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f6734b88640 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6bbf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 -- 172.21.15.33:0/2887732027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 msgr2=0x5650c6bbf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f6735389640 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bbf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bbf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 -- 172.21.15.33:0/2887732027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 msgr2=0x5650c6bbf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6bbf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 -- 172.21.15.33:0/2887732027 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650c6aac390 con 0x5650c6baa090 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f672ffff640 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f671c02c930 tx=0x7f671c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af68d22357d7a9e1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f671a7fc640 1 -- 172.21.15.33:0/2887732027 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c005000 con 0x5650c6baa090 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f671a7fc640 1 -- 172.21.15.33:0/2887732027 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671c005620 con 0x5650c6baa090 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.907+0000 7f671a7fc640 1 -- 172.21.15.33:0/2887732027 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c003cc0 con 0x5650c6baa090 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- 172.21.15.33:0/2887732027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 msgr2=0x5650c6bbad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f671c02c930 tx=0x7f671c049000 comp rx=0 tx=0).stop 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- 172.21.15.33:0/2887732027 shutdown_connections 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6baf200 0x5650c6bbf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baa090 0x5650c6bbad10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 --2- 172.21.15.33:0/2887732027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6ba96a0 0x5650c6bbf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- 172.21.15.33:0/2887732027 >> 172.21.15.33:0/2887732027 conn(0x5650c6ba4c10 msgr2=0x5650c6bab470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- 172.21.15.33:0/2887732027 shutdown_connections 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- 172.21.15.33:0/2887732027 wait complete. 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 -- start start 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.908+0000 7f67399b2740 1 Processor -- start 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6ba96a0 0x5650c6bb7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baa090 0x5650c6bb8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6bb9810 con 0x5650c6baf200 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6bb9980 con 0x5650c6baa090 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650c6bb9b40 con 0x5650c6ba96a0 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 10 client.0 _ll_register_callbacks cb 0x7f6724009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41180/0 (socket says 172.21.15.33:41180) 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 -- 172.21.15.33:0/3499128796 learned_addr learned my addr 172.21.15.33:0/3499128796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:12.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f672ffff640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baa090 0x5650c6bb8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6734b88640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6ba96a0 0x5650c6bb7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 -- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6ba96a0 msgr2=0x5650c6bb7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6ba96a0 0x5650c6bb7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 -- 172.21.15.33:0/3499128796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baa090 msgr2=0x5650c6bb8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baa090 0x5650c6bb8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 -- 172.21.15.33:0/3499128796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c049070 con 0x5650c6baf200 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f6735389640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6728096900 tx=0x7f6728099e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13978f9f384e785b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f670ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280cacb0 con 0x5650c6baf200 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f67280a9420 con 0x5650c6baf200 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672400ee80 con 0x5650c6baf200 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.909+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67280a90c0 con 0x5650c6baf200 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 10 client.15090 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:12.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5650c6ad1ee0 con 0x5650c6baf200 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67280b1080 con 0x5650c6baf200 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6728018030 con 0x5650c6baf200 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 10 client.15090 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 20 client.15090 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6c1b3f0 0x5650c6c17860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 4 client.15090 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 --> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650c68106c0 con 0x5650c6c1b3f0 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f6734b88640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6c1b3f0 0x5650c6c17860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f670ffff640 10 client.15090 ms_handle_connect on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.910+0000 7f6734b88640 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6c1b3f0 0x5650c6c17860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e0abcceb5b893585 server_cookie=1781106dd6b8289 in_seq=0 out_seq=0 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f670ffff640 1 -- 172.21.15.33:0/3499128796 <== mds.1 v2:172.21.15.152:6838/2298219777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5650c68106c0 con 0x5650c6c1b3f0 2024-01-25T16:07:12.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f670ffff640 10 client.15090 handle_command_reply: tid=0 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 client.15090 shutdown 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 msgr2=0x5650c6bb92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6728096900 tx=0x7f6728099e90 comp rx=0 tx=0).stop 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 shutdown_connections 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x5650c6c1b3f0 0x5650c6c17860 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650c6baf200 0x5650c6bb92d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650c6baa090 0x5650c6bb8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f670ffff640 0 client.15090 ms_handle_reset on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 --2- 172.21.15.33:0/3499128796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650c6ba96a0 0x5650c6bb7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 >> 172.21.15.33:0/3499128796 conn(0x5650c6ba4c10 msgr2=0x5650c6bbf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 shutdown_connections 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 1 -- 172.21.15.33:0/3499128796 wait complete. 2024-01-25T16:07:12.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.911+0000 7f67399b2740 20 client.15090 trim_cache size 0 max 16384 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24517, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 84.111573930999995, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 99.551426629000005 2024-01-25T16:07:12.926 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 msgr2=0x7f67200a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 0x7f67200a3830 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f671c005ab0 tx=0x7f671c02cbf0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 msgr2=0x7f673019e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f672800eaa0 tx=0x7f672800ef60 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 shutdown_connections 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f67200a1370 0x7f67200a3830 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6730102960 0x7f673019e260 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6730101fa0 0x7f6730197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 --2- 172.21.15.33:0/2132556382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67300fefd0 0x7f6730196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 >> 172.21.15.33:0/2132556382 conn(0x7f67300fa900 msgr2=0x7f67301053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 shutdown_connections 2024-01-25T16:07:12.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:12.912+0000 7f6736dec640 1 -- 172.21.15.33:0/2132556382 wait complete. 2024-01-25T16:07:12.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-25T16:07:13.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/106420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d54100d00 msgr2=0x7f1d54101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/106420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d54100d00 0x7f1d54101120 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009900 tx=0x7f1d4402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/106420463 shutdown_connections 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/106420463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d541028b0 0x7f1d54109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/106420463 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/106420463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d54100d00 0x7f1d54101120 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/106420463 >> 172.21.15.33:0/106420463 conn(0x7f1d540fc290 msgr2=0x7f1d540fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/106420463 shutdown_connections 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/106420463 wait complete. 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 Processor -- start 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.029+0000 7f1d5a0b6640 1 -- start start 2024-01-25T16:07:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 0x7f1d54196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5419e820 con 0x7f1d541028b0 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5419e990 con 0x7f1d54100d00 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5419eb50 con 0x7f1d54101ef0 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41204/0 (socket says 172.21.15.33:41204) 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 -- 172.21.15.33:0/3601773308 learned_addr learned my addr 172.21.15.33:0/3601773308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 0x7f1d54196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 -- 172.21.15.33:0/3601773308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 msgr2=0x7f1d54197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 -- 172.21.15.33:0/3601773308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 msgr2=0x7f1d54196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 0x7f1d54196d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 -- 172.21.15.33:0/3601773308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009560 con 0x7f1d541028b0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 0x7f1d54196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00cad0 tx=0x7f1d4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d50ff9640 1 -- 172.21.15.33:0/3601773308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00ae20 con 0x7f1d541028b0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d50ff9640 1 -- 172.21.15.33:0/3601773308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d4c005020 con 0x7f1d541028b0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d5419ee30 con 0x7f1d541028b0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d50ff9640 1 -- 172.21.15.33:0/3601773308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c00f6b0 con 0x7f1d541028b0 2024-01-25T16:07:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.030+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d5419f250 con 0x7f1d541028b0 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.032+0000 7f1d50ff9640 1 -- 172.21.15.33:0/3601773308 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f1d4c005190 con 0x7f1d541028b0 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 Processor -- start 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 -- start start 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d50ff9640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 0x7f1d3c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e933dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e9345690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e92ac030 con 0x5632e933e090 2024-01-25T16:07:13.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e92a9720 con 0x5632e933d6a0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e92a5010 con 0x5632e9343200 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 0x7f1d3c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d50ff9640 1 -- 172.21.15.33:0/3601773308 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d4c0ccee0 con 0x7f1d541028b0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e933dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41220/0 (socket says 172.21.15.33:41220) 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d52ffd640 1 -- 172.21.15.33:0/2550525386 learned_addr learned my addr 172.21.15.33:0/2550525386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.033+0000 7f1d53fff640 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e9345690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 0x7f1d3c0a3830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d48014470 tx=0x7f1d48014870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 -- 172.21.15.33:0/2550525386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 msgr2=0x5632e9345690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e9345690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 -- 172.21.15.33:0/2550525386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 msgr2=0x5632e9342b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 -- 172.21.15.33:0/2550525386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632e9267680 con 0x5632e933d6a0 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d537fe640 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e933dac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402c6b0 tx=0x7f1d44005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee7c0e796b482262 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d39ffb640 1 -- 172.21.15.33:0/2550525386 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d440046c0 con 0x5632e933d6a0 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d39ffb640 1 -- 172.21.15.33:0/2550525386 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d44004500 con 0x5632e933d6a0 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.034+0000 7f1d39ffb640 1 -- 172.21.15.33:0/2550525386 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d440054b0 con 0x5632e933d6a0 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- 172.21.15.33:0/2550525386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 msgr2=0x5632e933dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e933dac0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402c6b0 tx=0x7f1d44005e80 comp rx=0 tx=0).stop 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- 172.21.15.33:0/2550525386 shutdown_connections 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e9345690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e9342b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/2550525386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e933dac0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- 172.21.15.33:0/2550525386 >> 172.21.15.33:0/2550525386 conn(0x5632e9338c10 msgr2=0x5632e9339010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- 172.21.15.33:0/2550525386 shutdown_connections 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- 172.21.15.33:0/2550525386 wait complete. 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 -- start start 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.035+0000 7f1d5cc84740 1 Processor -- start 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e933e090 0x5632e9347d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 0x5632e934eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e925e5f0 con 0x5632e933d6a0 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e925e290 con 0x5632e9343200 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e9223300 con 0x5632e933e090 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 10 client.0 _ll_register_callbacks cb 0x5632e9259260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41234/0 (socket says 172.21.15.33:41234) 2024-01-25T16:07:13.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 -- 172.21.15.33:0/3792250476 learned_addr learned my addr 172.21.15.33:0/3792250476 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 0x5632e934eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 -- 172.21.15.33:0/3792250476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e933e090 msgr2=0x5632e9347d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e933e090 0x5632e9347d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 -- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 msgr2=0x5632e934eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 0x5632e934eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 -- 172.21.15.33:0/3792250476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d54109a40 con 0x5632e933d6a0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 0x5632e934eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1d44036040 tx=0x7f1d4402ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247561d39afe2a55 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44005640 con 0x5632e933d6a0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d4402ce40 con 0x5632e933d6a0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632e929dbc0 con 0x5632e933d6a0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d44004050 con 0x5632e933d6a0 2024-01-25T16:07:13.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 10 client.15102 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.036+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5632e92a3690 con 0x5632e933d6a0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632e929dbc0 con 0x5632e933d6a0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d4403c040 con 0x5632e933d6a0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 10 client.15102 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 10 client.15102 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 20 client.15102 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e9354620 0x5632e9353290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 4 client.15102 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632e8fa46c0 con 0x5632e9354620 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e9354620 0x5632e9353290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e9354620 0x5632e9353290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd6606069498c37 server_cookie=189c499796d62730 in_seq=0 out_seq=0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.037+0000 7f1d32ffd640 10 client.15102 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.038+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3792250476 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632e8fa46c0 con 0x5632e9354620 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.038+0000 7f1d32ffd640 10 client.15102 handle_command_reply: tid=0 2024-01-25T16:07:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.038+0000 7f1d5cc84740 1 client.15102 shutdown 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 msgr2=0x5632e9347820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1d44036040 tx=0x7f1d4402ccf0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 shutdown_connections 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e9354620 0x5632e9353290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e9343200 0x5632e934eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e933e090 0x5632e9347d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d32ffd640 0 client.15102 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3792250476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9347820 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 >> 172.21.15.33:0/3792250476 conn(0x5632e9338c10 msgr2=0x5632e9337da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 shutdown_connections 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3792250476 wait complete. 2024-01-25T16:07:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.039+0000 7f1d5cc84740 20 client.15102 trim_cache size 0 max 16384 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 Processor -- start 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 -- start start 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933e090 0x5632e934ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e93465d0 con 0x5632e933d6a0 2024-01-25T16:07:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e9355940 con 0x5632e933e090 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.054+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e9345d50 con 0x5632e9343200 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41242/0 (socket says 172.21.15.33:41242) 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 -- 172.21.15.33:0/1492467323 learned_addr learned my addr 172.21.15.33:0/1492467323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 -- 172.21.15.33:0/1492467323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 msgr2=0x5632e93537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 -- 172.21.15.33:0/1492467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933e090 msgr2=0x5632e934ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933e090 0x5632e934ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 -- 172.21.15.33:0/1492467323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632e9240390 con 0x5632e933d6a0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d537fe640 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402c170 tx=0x7f1d4402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe23eb3157f0a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d397fa640 1 -- 172.21.15.33:0/1492467323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44005440 con 0x5632e933d6a0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d397fa640 1 -- 172.21.15.33:0/1492467323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d44004460 con 0x5632e933d6a0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.055+0000 7f1d397fa640 1 -- 172.21.15.33:0/1492467323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44003c20 con 0x5632e933d6a0 2024-01-25T16:07:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- 172.21.15.33:0/1492467323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 msgr2=0x5632e9353290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d4402c170 tx=0x7f1d4402c5d0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- 172.21.15.33:0/1492467323 shutdown_connections 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933e090 0x5632e934ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/1492467323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933d6a0 0x5632e9353290 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- 172.21.15.33:0/1492467323 >> 172.21.15.33:0/1492467323 conn(0x5632e9338c10 msgr2=0x5632e933f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- 172.21.15.33:0/1492467323 shutdown_connections 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- 172.21.15.33:0/1492467323 wait complete. 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- start start 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 Processor -- start 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e934bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e934c910 con 0x5632e933e090 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e93a9aa0 con 0x5632e933d6a0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632e93a9c60 con 0x5632e9343200 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d5cc84740 10 client.0 _ll_register_callbacks cb 0x7f1d44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41250/0 (socket says 172.21.15.33:41250) 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.056+0000 7f1d52ffd640 1 -- 172.21.15.33:0/3300921102 learned_addr learned my addr 172.21.15.33:0/3300921102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d53fff640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e934bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 -- 172.21.15.33:0/3300921102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 msgr2=0x5632e93a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93a9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 -- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 msgr2=0x5632e934bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e934bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 -- 172.21.15.33:0/3300921102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632e92a3690 con 0x5632e933e090 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d52ffd640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5632e933eb00 tx=0x7f1d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a37ad9731f8ab7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d48011bb0 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f1d48011d10 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632e929dea0 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d48011ed0 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 10 client.15108 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5632e9265ee0 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632e929dea0 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d4800b030 con 0x5632e933e090 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 10 client.15108 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 10 client.15108 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 20 client.15108 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e93af030 0x5632e93ab4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 4 client.15108 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:13.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.057+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5632e8ff7d10 con 0x5632e93af030 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e93af030 0x5632e93ab4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d537fe640 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e93af030 0x5632e93ab4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba86a13d4f6fe0cc server_cookie=8c2d09b290fd806d in_seq=0 out_seq=0 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d32ffd640 10 client.15108 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d32ffd640 1 -- 172.21.15.33:0/3300921102 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5632e8ff7d10 con 0x5632e93af030 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d32ffd640 10 client.15108 handle_command_reply: tid=0 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d5cc84740 1 client.15108 shutdown 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 msgr2=0x5632e934c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x5632e933eb00 tx=0x7f1d480046c0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.058+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 shutdown_connections 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632e93af030 0x5632e93ab4d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632e9343200 0x5632e93a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d32ffd640 0 client.15108 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632e933e090 0x5632e934c210 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 --2- 172.21.15.33:0/3300921102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632e933d6a0 0x5632e934bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 >> 172.21.15.33:0/3300921102 conn(0x5632e9338c10 msgr2=0x5632e933f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 shutdown_connections 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 1 -- 172.21.15.33:0/3300921102 wait complete. 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5cc84740 20 client.15108 trim_cache size 0 max 16384 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24536, 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 86.275211630000001, 2024-01-25T16:07:13.070 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:13.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 91.423970412000003 2024-01-25T16:07:13.071 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 msgr2=0x7f1d3c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 0x7f1d3c0a3830 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1d48014470 tx=0x7f1d48014870 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 msgr2=0x7f1d5419e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00cad0 tx=0x7f1d4c00cf90 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 shutdown_connections 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f1d3c0a1370 0x7f1d3c0a3830 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d541028b0 0x7f1d5419e2b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1d54101ef0 0x7f1d54197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 --2- 172.21.15.33:0/3601773308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1d54100d00 0x7f1d54196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 >> 172.21.15.33:0/3601773308 conn(0x7f1d540fc290 msgr2=0x7f1d54105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 shutdown_connections 2024-01-25T16:07:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:13.059+0000 7f1d5a0b6640 1 -- 172.21.15.33:0/3601773308 wait complete. 2024-01-25T16:07:13.073 INFO:tasks.cephadm:Setup complete, yielding 2024-01-25T16:07:13.074 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-25T16:07:13.084 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:07:13.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph orch status' 2024-01-25T16:07:13.326 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:13.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:13 smithi033 ceph-mon[31417]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:07:13.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:13 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2707165488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:13.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:13 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3144170252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:13 smithi152 ceph-mon[36015]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:07:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:13 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2707165488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:13 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3144170252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:13 smithi076 ceph-mon[35642]: pgmap v330: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-01-25T16:07:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:13 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2707165488' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:13 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3144170252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:14.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 -- 172.21.15.33:0/416683467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 msgr2=0x7f5b50100b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 --2- 172.21.15.33:0/416683467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b50100b20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b440099c0 tx=0x7f5b44009cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 -- 172.21.15.33:0/416683467 shutdown_connections 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 --2- 172.21.15.33:0/416683467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b50108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 --2- 172.21.15.33:0/416683467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b50100b20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 --2- 172.21.15.33:0/416683467 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 0x7f5b50101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 -- 172.21.15.33:0/416683467 >> 172.21.15.33:0/416683467 conn(0x7f5b500fbca0 msgr2=0x7f5b500fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 -- 172.21.15.33:0/416683467 shutdown_connections 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.889+0000 7f5b571df700 1 -- 172.21.15.33:0/416683467 wait complete. 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 Processor -- start 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 -- start start 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 0x7f5b50197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:14.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b5019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5019f160 con 0x7f5b50100700 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5019f2a0 con 0x7f5b501022a0 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b571df700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5019f430 con 0x7f5b50101900 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41262/0 (socket says 172.21.15.33:41262) 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 -- 172.21.15.33:0/2268059588 learned_addr learned my addr 172.21.15.33:0/2268059588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:14.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b4ffff700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 0x7f5b50197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b5577c700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b5019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 -- 172.21.15.33:0/2268059588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 msgr2=0x7f5b50197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 0x7f5b50197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 -- 172.21.15.33:0/2268059588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 msgr2=0x7f5b5019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b5019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b54f7b700 1 -- 172.21.15.33:0/2268059588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440095f0 con 0x7f5b50100700 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.890+0000 7f5b5577c700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b5019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b54f7b700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b440036f0 tx=0x7f5b440047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4401d070 con 0x7f5b50100700 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5019f6b0 con 0x7f5b50100700 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b440041f0 con 0x7f5b50100700 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b4400f460 con 0x7f5b50100700 2024-01-25T16:07:14.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.891+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5019fb50 con 0x7f5b50100700 2024-01-25T16:07:14.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.892+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b500684a0 con 0x7f5b50100700 2024-01-25T16:07:14.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.893+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7f5b44022650 con 0x7f5b50100700 2024-01-25T16:07:14.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.893+0000 7f5b4dffb700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 0x7f5b400a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:14.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.894+0000 7f5b4ffff700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 0x7f5b400a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:14.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.894+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5b440ce140 con 0x7f5b50100700 2024-01-25T16:07:14.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.894+0000 7f5b4ffff700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 0x7f5b400a38c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00ac50 tx=0x7f5b3c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:14.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:14.898+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f5b440974f0 con 0x7f5b50100700 2024-01-25T16:07:15.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.120+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b500fddf0 con 0x7f5b400a1400 2024-01-25T16:07:15.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.121+0000 7f5b4dffb700 1 -- 172.21.15.33:0/2268059588 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5b500fddf0 con 0x7f5b400a1400 2024-01-25T16:07:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-01-25T16:07:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-01-25T16:07:15.124 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-01-25T16:07:15.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 msgr2=0x7f5b400a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 0x7f5b400a38c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00ac50 tx=0x7f5b3c00a380 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 msgr2=0x7f5b501975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b440036f0 tx=0x7f5b440047a0 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 shutdown_connections 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f5b400a1400 0x7f5b400a38c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5b501022a0 0x7f5b5019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b50100700 0x7f5b501975d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 --2- 172.21.15.33:0/2268059588 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5b50101900 0x7f5b50197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 >> 172.21.15.33:0/2268059588 conn(0x7f5b500fbca0 msgr2=0x7f5b500fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 shutdown_connections 2024-01-25T16:07:15.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:15.123+0000 7f5b571df700 1 -- 172.21.15.33:0/2268059588 wait complete. 2024-01-25T16:07:15.502 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:15 smithi033 ceph-mon[31417]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:15.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:15 smithi152 ceph-mon[36015]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:15 smithi076 ceph-mon[35642]: pgmap v331: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:15.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph orch ps' 2024-01-25T16:07:16.083 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:16.531 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:16 smithi033 ceph-mon[31417]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:16.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:16 smithi152 ceph-mon[36015]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:16.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:16 smithi076 ceph-mon[35642]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:17.480 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:17 smithi033 ceph-mon[31417]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:17 smithi152 ceph-mon[36015]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:17 smithi076 ceph-mon[35642]: pgmap v332: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-01-25T16:07:17.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/3120875420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 msgr2=0x7fd4bc100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:17.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/3120875420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc100e00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009990 tx=0x7fd4a8009ca0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/3120875420 shutdown_connections 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/3120875420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 0x7fd4bc108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/3120875420 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 0x7fd4bc102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/3120875420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc100e00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/3120875420 >> 172.21.15.33:0/3120875420 conn(0x7fd4bc0fb450 msgr2=0x7fd4bc0fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/3120875420 shutdown_connections 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/3120875420 wait complete. 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 Processor -- start 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.721+0000 7fd4c2fc8700 1 -- start start 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 0x7fd4bc197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 0x7fd4bc19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc19f280 con 0x7fd4bc1009a0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc19f3c0 con 0x7fd4bc101340 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4bc19f550 con 0x7fd4bc101cf0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41272/0 (socket says 172.21.15.33:41272) 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 -- 172.21.15.33:0/1607281992 learned_addr learned my addr 172.21.15.33:0/1607281992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4bbfff700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 0x7fd4bc197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c1565700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 0x7fd4bc19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 -- 172.21.15.33:0/1607281992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 msgr2=0x7fd4bc19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 0x7fd4bc19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 -- 172.21.15.33:0/1607281992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 msgr2=0x7fd4bc197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 0x7fd4bc197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 -- 172.21.15.33:0/1607281992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a80095f0 con 0x7fd4bc1009a0 2024-01-25T16:07:17.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c0d64700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d9b0 tx=0x7fd4b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000b950 con 0x7fd4bc1009a0 2024-01-25T16:07:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4bc19f830 con 0x7fd4bc1009a0 2024-01-25T16:07:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4b0007620 con 0x7fd4bc1009a0 2024-01-25T16:07:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.722+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b000f3c0 con 0x7fd4bc1009a0 2024-01-25T16:07:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.723+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4bc19fd30 con 0x7fd4bc1009a0 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.724+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4bc0684a0 con 0x7fd4bc1009a0 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.724+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 135946+0+0 (secure 0 0 0) 0x7fd4b0010460 con 0x7fd4bc1009a0 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.725+0000 7fd4b9ffb700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 0x7fd4ac0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.725+0000 7fd4bbfff700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 0x7fd4ac0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.725+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd4b00cca70 con 0x7fd4bc1009a0 2024-01-25T16:07:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.726+0000 7fd4bbfff700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 0x7fd4ac0a38c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b5c0 tx=0x7fd4a8003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:17.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.730+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd4b0095d90 con 0x7fd4bc1009a0 2024-01-25T16:07:17.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.936+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4bc0fd650 con 0x7fd4ac0a1400 2024-01-25T16:07:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.950+0000 7fd4b9ffb700 1 -- 172.21.15.33:0/1607281992 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7fd4bc0fd650 con 0x7fd4ac0a1400 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (99s) 17s ago 98s 13.8M - 19.0.0-885-g661946e7 0b5dc47a38d7 71596c75ea09 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi076 running (96s) 30s ago 96s 26.1M - 19.0.0-885-g661946e7 0b5dc47a38d7 6ceb75aa7c35 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi152 running (110s) 30s ago 110s 20.0M - 19.0.0-885-g661946e7 0b5dc47a38d7 786824a515c1 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (118s) 17s ago 118s 13.5M - 19.0.0-885-g661946e7 0b5dc47a38d7 1e52c9ce1b9d 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi076 running (116s) 30s ago 115s 16.6M - 19.0.0-885-g661946e7 0b5dc47a38d7 fd41a3180d5c 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi152 running (113s) 30s ago 113s 18.1M - 19.0.0-885-g661946e7 0b5dc47a38d7 3b309ea37af6 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (102s) 17s ago 101s 15.2M - 19.0.0-885-g661946e7 0b5dc47a38d7 cf895c949215 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi076 running (107s) 30s ago 107s 19.8M - 19.0.0-885-g661946e7 0b5dc47a38d7 08e91dc07457 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi152 running (104s) 30s ago 104s 18.8M - 19.0.0-885-g661946e7 0b5dc47a38d7 b1f77793b5a2 2024-01-25T16:07:17.954 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (2m) 17s ago 2m 16.3M - 19.0.0-885-g661946e7 0b5dc47a38d7 369c7acc46fa 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi076 running (2m) 30s ago 2m 13.9M - 19.0.0-885-g661946e7 0b5dc47a38d7 70ede9c7e35f 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi152 running (2m) 30s ago 2m 13.9M - 19.0.0-885-g661946e7 0b5dc47a38d7 81d08ae07be9 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (9m) 17s ago 9m 505M - 19.0.0-885-g661946e7 0b5dc47a38d7 ea9ea84c7789 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi076 *:8443,8765 running (7m) 30s ago 7m 443M - 19.0.0-885-g661946e7 0b5dc47a38d7 015b1b48e706 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi152 *:8443,8765 running (7m) 30s ago 7m 442M - 19.0.0-885-g661946e7 0b5dc47a38d7 249c52d1598d 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (9m) 17s ago 9m 50.0M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 6e997038abfe 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi076 running (7m) 30s ago 7m 38.8M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 61742daf75f1 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi152 running (7m) 30s ago 7m 39.9M 2048M 19.0.0-885-g661946e7 0b5dc47a38d7 3d368563f5e3 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (6m) 17s ago 6m 55.8M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 d36bd42a6be6 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi076 running (6m) 30s ago 6m 56.3M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 629705f7c28f 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi152 running (6m) 30s ago 6m 57.4M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 89688a3f56e3 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (5m) 17s ago 5m 54.1M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 a2dade741952 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi076 running (5m) 30s ago 5m 45.8M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 e99e859568f3 2024-01-25T16:07:17.955 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi152 running (4m) 30s ago 4m 54.0M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 069d1930cdd0 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (4m) 17s ago 4m 47.5M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 7c191be84bcd 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi076 running (4m) 30s ago 4m 53.3M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 fd91977a95db 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi152 running (3m) 30s ago 3m 55.8M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 05ca1baaf031 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (3m) 17s ago 3m 46.6M 4273M 19.0.0-885-g661946e7 0b5dc47a38d7 5967688b2b06 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi076 running (2m) 30s ago 2m 46.3M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 c207b4d9172a 2024-01-25T16:07:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi152 running (2m) 30s ago 2m 43.7M 4270M 19.0.0-885-g661946e7 0b5dc47a38d7 7c7e3631730c 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 msgr2=0x7fd4ac0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 0x7fd4ac0a38c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800b5c0 tx=0x7fd4a8003790 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 msgr2=0x7fd4bc1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000d9b0 tx=0x7fd4b000dcc0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 shutdown_connections 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd4ac0a1400 0x7fd4ac0a38c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4bc101cf0 0x7fd4bc19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4bc1009a0 0x7fd4bc1976f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 --2- 172.21.15.33:0/1607281992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4bc101340 0x7fd4bc197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 >> 172.21.15.33:0/1607281992 conn(0x7fd4bc0fb450 msgr2=0x7fd4bc0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 shutdown_connections 2024-01-25T16:07:17.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:17.952+0000 7fd4c2fc8700 1 -- 172.21.15.33:0/1607281992 wait complete. 2024-01-25T16:07:18.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph orch ls' 2024-01-25T16:07:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:19.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:19 smithi033 ceph-mon[31417]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:19.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:19 smithi033 ceph-mon[31417]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:19 smithi152 ceph-mon[36015]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:19 smithi152 ceph-mon[36015]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:19 smithi076 ceph-mon[35642]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:19 smithi076 ceph-mon[35642]: pgmap v333: 97 pgs: 97 active+clean; 636 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-01-25T16:07:20.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:20 smithi033 ceph-mon[31417]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-01-25T16:07:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 -- 172.21.15.33:0/2088908124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784101810 msgr2=0x7f7784101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 --2- 172.21.15.33:0/2088908124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784101810 0x7f7784101c30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f77740099c0 tx=0x7f7774009cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 -- 172.21.15.33:0/2088908124 shutdown_connections 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 --2- 172.21.15.33:0/2088908124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f77841033b0 0x7f7784109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 --2- 172.21.15.33:0/2088908124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784101810 0x7f7784101c30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 --2- 172.21.15.33:0/2088908124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7784102a10 0x7f7784102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.363+0000 7f778c1f8700 1 -- 172.21.15.33:0/2088908124 >> 172.21.15.33:0/2088908124 conn(0x7f77840fcdf0 msgr2=0x7f77840ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- 172.21.15.33:0/2088908124 shutdown_connections 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- 172.21.15.33:0/2088908124 wait complete. 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 Processor -- start 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- start start 2024-01-25T16:07:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7784101810 0x7f7784075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77841033b0 0x7f7784074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7784074890 con 0x7f7784102a10 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7784074a00 con 0x7f77841033b0 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.364+0000 7f778c1f8700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77841a17e0 con 0x7f7784101810 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42484/0 (socket says 172.21.15.33:42484) 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 -- 172.21.15.33:0/3729669262 learned_addr learned my addr 172.21.15.33:0/3729669262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789f94700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7784101810 0x7f7784075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f778a795700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77841033b0 0x7f7784074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 -- 172.21.15.33:0/3729669262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7784101810 msgr2=0x7f7784075580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7784101810 0x7f7784075580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 -- 172.21.15.33:0/3729669262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77841033b0 msgr2=0x7f7784074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77841033b0 0x7f7784074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 -- 172.21.15.33:0/3729669262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77740095f0 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f7789793700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f778000d950 tx=0x7f778000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000f890 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7780010460 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778000e610 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77841a1ac0 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.365+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77841a2010 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.367+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7784108cf0 con 0x7f7784102a10 2024-01-25T16:07:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.368+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f77800108d0 con 0x7f7784102a10 2024-01-25T16:07:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.368+0000 7f777affd700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 0x7f77700a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.368+0000 7f7789f94700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 0x7f77700a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.368+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f77800cc850 con 0x7f7784102a10 2024-01-25T16:07:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.369+0000 7f7789f94700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 0x7f77700a38b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7774003820 tx=0x7f7774005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.373+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7780095cd0 con 0x7f7784102a10 2024-01-25T16:07:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.581+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f778402d540 con 0x7f77700a13f0 2024-01-25T16:07:20.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.587+0000 7f777affd700 1 -- 172.21.15.33:0/3729669262 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f778402d540 con 0x7f77700a13f0 2024-01-25T16:07:20.590 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-01-25T16:07:20.590 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 33s ago 2m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi076=b;smithi076=e;smithi076=h;smithi076=k;smithi152=c;smithi152=f;smithi152=i;smithi152=l;count:12 2024-01-25T16:07:20.590 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 33s ago 7m smithi033=x;smithi076=y;smithi152=z;count:3 2024-01-25T16:07:20.590 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 33s ago 7m smithi033:172.21.15.33=a;smithi076:172.21.15.76=b;smithi152:172.21.15.152=c;count:3 2024-01-25T16:07:20.590 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 33s ago - 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 msgr2=0x7f77700a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 0x7f77700a38b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7774003820 tx=0x7f7774005ad0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 msgr2=0x7f7784073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f778000d950 tx=0x7f778000dc60 comp rx=0 tx=0).stop 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 shutdown_connections 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f77700a13f0 0x7f77700a38b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77841033b0 0x7f7784074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7784101810 0x7f7784075580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 --2- 172.21.15.33:0/3729669262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7784102a10 0x7f7784073bd0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 >> 172.21.15.33:0/3729669262 conn(0x7f77840fcdf0 msgr2=0x7f7784105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 shutdown_connections 2024-01-25T16:07:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:20.589+0000 7f778c1f8700 1 -- 172.21.15.33:0/3729669262 wait complete. 2024-01-25T16:07:20.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:20 smithi152 ceph-mon[36015]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-01-25T16:07:20.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:20 smithi076 ceph-mon[35642]: mgrmap e18: x(active, since 8m), standbys: y, z 2024-01-25T16:07:21.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph orch host ls' 2024-01-25T16:07:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:21 smithi033 ceph-mon[31417]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:21.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:21 smithi152 ceph-mon[36015]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:21 smithi076 ceph-mon[35642]: pgmap v334: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:22.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:22 smithi033 ceph-mon[31417]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:22 smithi152 ceph-mon[36015]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:22.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:22 smithi076 ceph-mon[35642]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 -- 172.21.15.33:0/426849299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34102af0 msgr2=0x7fcc34102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 --2- 172.21.15.33:0/426849299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34102af0 0x7fcc34102f50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcc28009990 tx=0x7fcc28009ca0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 -- 172.21.15.33:0/426849299 shutdown_connections 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 --2- 172.21.15.33:0/426849299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34103490 0x7fcc34109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 --2- 172.21.15.33:0/426849299 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc340fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 --2- 172.21.15.33:0/426849299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34102af0 0x7fcc34102f50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 -- 172.21.15.33:0/426849299 >> 172.21.15.33:0/426849299 conn(0x7fcc340fb470 msgr2=0x7fcc340fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 -- 172.21.15.33:0/426849299 shutdown_connections 2024-01-25T16:07:23.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.220+0000 7fcc3887f700 1 -- 172.21.15.33:0/426849299 wait complete. 2024-01-25T16:07:23.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.221+0000 7fcc3887f700 1 Processor -- start 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.221+0000 7fcc3887f700 1 -- start start 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc34197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34102af0 0x7fcc34197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419f220 con 0x7fcc34103490 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419f360 con 0x7fcc34102af0 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3419f4f0 con 0x7fcc340ffb10 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42508/0 (socket says 172.21.15.33:42508) 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 -- 172.21.15.33:0/475129246 learned_addr learned my addr 172.21.15.33:0/475129246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc31d9b700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34102af0 0x7fcc34197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3259c700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc34197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:23.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 -- 172.21.15.33:0/475129246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 msgr2=0x7fcc34197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc34197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 -- 172.21.15.33:0/475129246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34102af0 msgr2=0x7fcc34197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34102af0 0x7fcc34197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 -- 172.21.15.33:0/475129246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280095f0 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3259c700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc34197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc32d9d700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00add0 tx=0x7fcc2c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c00f070 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3419f7d0 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc2c007b30 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc2c005310 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.222+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc3419fcd0 con 0x7fcc34103490 2024-01-25T16:07:23.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.223+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc34108d80 con 0x7fcc34103490 2024-01-25T16:07:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.224+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fcc2c00c940 con 0x7fcc34103490 2024-01-25T16:07:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.225+0000 7fcc237fe700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 0x7fcc1c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.225+0000 7fcc3259c700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 0x7fcc1c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.225+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcc2c0cc760 con 0x7fcc34103490 2024-01-25T16:07:23.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.225+0000 7fcc3259c700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 0x7fcc1c0a3830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:23.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.229+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fcc2c095c70 con 0x7fcc34103490 2024-01-25T16:07:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.438+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc3402d4d0 con 0x7fcc1c0a1370 2024-01-25T16:07:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.439+0000 7fcc237fe700 1 -- 172.21.15.33:0/475129246 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fcc3402d4d0 con 0x7fcc1c0a1370 2024-01-25T16:07:23.442 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-01-25T16:07:23.442 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-01-25T16:07:23.443 INFO:teuthology.orchestra.run.smithi033.stdout:smithi076 172.21.15.76 2024-01-25T16:07:23.443 INFO:teuthology.orchestra.run.smithi033.stdout:smithi152 172.21.15.152 2024-01-25T16:07:23.443 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-01-25T16:07:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 msgr2=0x7fcc1c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:23.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 0x7fcc1c0a3830 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fcc24005fd0 tx=0x7fcc24005f00 comp rx=0 tx=0).stop 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 msgr2=0x7fcc3419ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c00add0 tx=0x7fcc2c00c330 comp rx=0 tx=0).stop 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 shutdown_connections 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fcc1c0a1370 0x7fcc1c0a3830 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc34103490 0x7fcc3419ec50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc340ffb10 0x7fcc34197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 --2- 172.21.15.33:0/475129246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc34102af0 0x7fcc34197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 >> 172.21.15.33:0/475129246 conn(0x7fcc340fb470 msgr2=0x7fcc34105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 shutdown_connections 2024-01-25T16:07:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:23.441+0000 7fcc3887f700 1 -- 172.21.15.33:0/475129246 wait complete. 2024-01-25T16:07:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:23 smithi033 ceph-mon[31417]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:23 smithi152 ceph-mon[36015]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:23 smithi076 ceph-mon[35642]: pgmap v335: 97 pgs: 97 active+clean; 636 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:24.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph orch device ls' 2024-01-25T16:07:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:25.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:25 smithi033 ceph-mon[31417]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:25.557 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:25 smithi033 ceph-mon[31417]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:25 smithi152 ceph-mon[36015]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:25 smithi152 ceph-mon[36015]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:25 smithi076 ceph-mon[35642]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:25 smithi076 ceph-mon[35642]: pgmap v336: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:26.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.034+0000 7fe6f827b700 1 -- 172.21.15.33:0/656446750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0101900 msgr2=0x7fe6f0101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:26.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- 172.21.15.33:0/656446750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0101900 0x7fe6f0101d60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40099c0 tx=0x7fe6e4009cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- 172.21.15.33:0/656446750 shutdown_connections 2024-01-25T16:07:26.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- 172.21.15.33:0/656446750 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f01022a0 0x7fe6f0108b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- 172.21.15.33:0/656446750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 0x7fe6f0100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- 172.21.15.33:0/656446750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f0101900 0x7fe6f0101d60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- 172.21.15.33:0/656446750 >> 172.21.15.33:0/656446750 conn(0x7fe6f00fbca0 msgr2=0x7fe6f00fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- 172.21.15.33:0/656446750 shutdown_connections 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- 172.21.15.33:0/656446750 wait complete. 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 Processor -- start 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- start start 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 0x7fe6f0197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0101900 0x7fe6f0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019f220 con 0x7fe6f01022a0 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019f360 con 0x7fe6f0100700 2024-01-25T16:07:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.035+0000 7fe6f827b700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f019f4f0 con 0x7fe6f0101900 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42526/0 (socket says 172.21.15.33:42526) 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 -- 172.21.15.33:0/1607166398 learned_addr learned my addr 172.21.15.33:0/1607166398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f5816700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0101900 0x7fe6f0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6017700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 0x7fe6f0197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 -- 172.21.15.33:0/1607166398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0101900 msgr2=0x7fe6f0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0101900 0x7fe6f0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 -- 172.21.15.33:0/1607166398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 msgr2=0x7fe6f0197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 0x7fe6f0197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 -- 172.21.15.33:0/1607166398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40095f0 con 0x7fe6f01022a0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f6818700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007da0 tx=0x7fe6e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000ad00 con 0x7fe6f01022a0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f019f7d0 con 0x7fe6f01022a0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e000ae60 con 0x7fe6f01022a0 2024-01-25T16:07:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0011660 con 0x7fe6f01022a0 2024-01-25T16:07:26.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.036+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f019fc80 con 0x7fe6f01022a0 2024-01-25T16:07:26.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.037+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f00684a0 con 0x7fe6f01022a0 2024-01-25T16:07:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.038+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe6e0012420 con 0x7fe6f01022a0 2024-01-25T16:07:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.039+0000 7fe6ef7fe700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 0x7fe6d80a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.039+0000 7fe6f6017700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 0x7fe6d80a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.039+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe6e00ccdb0 con 0x7fe6f01022a0 2024-01-25T16:07:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.039+0000 7fe6f6017700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 0x7fe6d80a3830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.043+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe6e0096230 con 0x7fe6f01022a0 2024-01-25T16:07:26.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.253+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6f006a650 con 0x7fe6d80a1370 2024-01-25T16:07:26.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.255+0000 7fe6ef7fe700 1 -- 172.21.15.33:0/1607166398 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fe6f006a650 con 0x7fe6d80a1370 2024-01-25T16:07:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-01-25T16:07:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-01-25T16:07:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:smithi076 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400U4400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-01-25T16:07:26.258 INFO:teuthology.orchestra.run.smithi033.stdout:smithi152 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JT400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 msgr2=0x7fe6d80a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 0x7fe6d80a3830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8006fd0 tx=0x7fe6e8008040 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 msgr2=0x7fe6f019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0007da0 tx=0x7fe6e000a340 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 shutdown_connections 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe6d80a1370 0x7fe6d80a3830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6f01022a0 0x7fe6f019ec50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6f0100700 0x7fe6f0197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 --2- 172.21.15.33:0/1607166398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe6f0101900 0x7fe6f0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 >> 172.21.15.33:0/1607166398 conn(0x7fe6f00fbca0 msgr2=0x7fe6f0069f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 shutdown_connections 2024-01-25T16:07:26.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:26.258+0000 7fe6f827b700 1 -- 172.21.15.33:0/1607166398 wait complete. 2024-01-25T16:07:27.003 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-01-25T16:07:27.012 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:07:27.013 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph fs dump' 2024-01-25T16:07:27.247 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:27 smithi033 ceph-mon[31417]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:27.247 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:27 smithi033 ceph-mon[31417]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:27.262 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:27 smithi152 ceph-mon[36015]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:27 smithi152 ceph-mon[36015]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:27 smithi076 ceph-mon[35642]: pgmap v337: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:27 smithi076 ceph-mon[35642]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 -- 172.21.15.33:0/420613752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102a40 msgr2=0x7f2ce8102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 --2- 172.21.15.33:0/420613752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102a40 0x7f2ce8102ea0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00099c0 tx=0x7f2cd0009cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 -- 172.21.15.33:0/420613752 shutdown_connections 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 --2- 172.21.15.33:0/420613752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce81033e0 0x7f2ce8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 --2- 172.21.15.33:0/420613752 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce8101840 0x7f2ce8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 --2- 172.21.15.33:0/420613752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8102a40 0x7f2ce8102ea0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 -- 172.21.15.33:0/420613752 >> 172.21.15.33:0/420613752 conn(0x7f2ce80fcde0 msgr2=0x7f2ce80ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 -- 172.21.15.33:0/420613752 shutdown_connections 2024-01-25T16:07:29.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.014+0000 7f2ceda59700 1 -- 172.21.15.33:0/420613752 wait complete. 2024-01-25T16:07:29.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 Processor -- start 2024-01-25T16:07:29.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 -- start start 2024-01-25T16:07:29.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:29.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce8102a40 0x7f2ce8073bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce81033e0 0x7f2ce8074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8074890 con 0x7f2ce8101840 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8074a00 con 0x7f2ce8102a40 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ceda59700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81a1690 con 0x7f2ce81033e0 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ce6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ce6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44964/0 (socket says 172.21.15.33:44964) 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ce6ffd700 1 -- 172.21.15.33:0/2883930728 learned_addr learned my addr 172.21.15.33:0/2883930728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ce67fc700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce8102a40 0x7f2ce8073bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.015+0000 7f2ce77fe700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce81033e0 0x7f2ce8074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 -- 172.21.15.33:0/2883930728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce81033e0 msgr2=0x7f2ce8074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce81033e0 0x7f2ce8074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 -- 172.21.15.33:0/2883930728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce8102a40 msgr2=0x7f2ce8073bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:29.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce8102a40 0x7f2ce8073bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 -- 172.21.15.33:0/2883930728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd00095f0 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ce6ffd700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007e00 tx=0x7f2ce000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce000ac90 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce81a1970 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ce000adf0 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0011600 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.016+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81a1ec0 con 0x7f2ce8101840 2024-01-25T16:07:29.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.017+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8108ba0 con 0x7f2ce8101840 2024-01-25T16:07:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.018+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2ce0003a00 con 0x7f2ce8101840 2024-01-25T16:07:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.019+0000 7f2ceca57700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 0x7f2cd40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.019+0000 7f2ce67fc700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 0x7f2cd40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.019+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ce00cce40 con 0x7f2ce8101840 2024-01-25T16:07:29.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.019+0000 7f2ce67fc700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 0x7f2cd40a3830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009990 tx=0x7f2cd0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.022+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f2ce0096280 con 0x7f2ce8101840 2024-01-25T16:07:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.269+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ce810d040 con 0x7f2ce8101840 2024-01-25T16:07:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.270+0000 7f2ceca57700 1 -- 172.21.15.33:0/2883930728 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+2939 (secure 0 0 0) 0x7f2ce0016b90 con 0x7f2ce8101840 2024-01-25T16:07:29.272 INFO:teuthology.orchestra.run.smithi033.stdout:e21 2024-01-25T16:07:29.272 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-01-25T16:05:46.107985+0000 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-01-25T16:05:50.161531+0000 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-01-25T16:07:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 3 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24536,1=24517,2=24494} 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-01-25T16:07:29.274 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:24536} state up:active seq 3 addr [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{0:24493} state up:standby-replay seq 1 addr [v2:172.21.15.152:6836/2179148160,v1:172.21.15.152:6837/2179148160] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{1:24517} state up:active seq 5 addr [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{1:24484} state up:standby-replay seq 1 addr [v2:172.21.15.152:6834/1042830191,v1:172.21.15.152:6835/1042830191] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{2:24494} state up:active seq 6 addr [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{2:24464} state up:standby-replay seq 1 addr [v2:172.21.15.76:6834/3469230010,v1:172.21.15.76:6835/3469230010] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14562} state up:standby seq 1 addr [v2:172.21.15.33:6834/2241105736,v1:172.21.15.33:6835/2241105736] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14574} state up:standby seq 1 addr [v2:172.21.15.33:6836/2646618640,v1:172.21.15.33:6837/2646618640] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{-1:14637} state up:standby seq 1 addr [v2:172.21.15.33:6838/3604277754,v1:172.21.15.33:6839/3604277754] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{-1:14649} state up:standby seq 1 addr [v2:172.21.15.33:6840/3585610078,v1:172.21.15.33:6841/3585610078] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.276 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{-1:24424} state up:standby seq 1 addr [v2:172.21.15.152:6832/3591759228,v1:172.21.15.152:6833/3591759228] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.276 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{-1:24437} state up:standby seq 1 addr [v2:172.21.15.76:6832/3364885054,v1:172.21.15.76:6833/3364885054] compat {c=[1],r=[1],i=[fff]}] 2024-01-25T16:07:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 msgr2=0x7f2cd40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 0x7f2cd40a3830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0009990 tx=0x7f2cd0000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 msgr2=0x7f2ce8075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0007e00 tx=0x7f2ce000a340 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 shutdown_connections 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2cd40a1370 0x7f2cd40a3830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2ce81033e0 0x7f2ce8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce8101840 0x7f2ce8075580 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 --2- 172.21.15.33:0/2883930728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ce8102a40 0x7f2ce8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 >> 172.21.15.33:0/2883930728 conn(0x7f2ce80fcde0 msgr2=0x7f2ce8105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 shutdown_connections 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:29.272+0000 7f2ceda59700 1 -- 172.21.15.33:0/2883930728 wait complete. 2024-01-25T16:07:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:29 smithi033 ceph-mon[31417]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:29.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:29 smithi152 ceph-mon[36015]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:29.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:29 smithi076 ceph-mon[35642]: pgmap v338: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-01-25T16:07:29.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- bash -c 'ceph osd dump' 2024-01-25T16:07:30.160 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:30 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2883930728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-25T16:07:30.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:30 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2883930728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-25T16:07:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:30 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2883930728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-01-25T16:07:31.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:31 smithi033 ceph-mon[31417]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:31.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:31 smithi152 ceph-mon[36015]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:31 smithi076 ceph-mon[35642]: pgmap v339: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 -- 172.21.15.33:0/867727378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d41075a0 msgr2=0x7f08d4109990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/867727378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d41075a0 0x7f08d4109990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00b0a0 tx=0x7f08cc00b3b0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 -- 172.21.15.33:0/867727378 shutdown_connections 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/867727378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d41075a0 0x7f08d4109990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/867727378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4102340 0x7f08d4104730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/867727378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d4104c70 0x7f08d4107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.764+0000 7f08d9e9a700 1 -- 172.21.15.33:0/867727378 >> 172.21.15.33:0/867727378 conn(0x7f08d40fbcd0 msgr2=0x7f08d40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 -- 172.21.15.33:0/867727378 shutdown_connections 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 -- 172.21.15.33:0/867727378 wait complete. 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 Processor -- start 2024-01-25T16:07:31.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 -- start start 2024-01-25T16:07:31.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:31.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.765+0000 7f08d9e9a700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4104c70 0x7f08d4197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d41075a0 0x7f08d419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f3b0 con 0x7f08d4102340 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f4f0 con 0x7f08d41075a0 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f680 con 0x7f08d4104c70 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44984/0 (socket says 172.21.15.33:44984) 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 -- 172.21.15.33:0/2015264434 learned_addr learned my addr 172.21.15.33:0/2015264434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d2ffd700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4104c70 0x7f08d4197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d3fff700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d41075a0 0x7f08d419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 -- 172.21.15.33:0/2015264434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4104c70 msgr2=0x7f08d4197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4104c70 0x7f08d4197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 -- 172.21.15.33:0/2015264434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d41075a0 msgr2=0x7f08d419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:31.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d41075a0 0x7f08d419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 -- 172.21.15.33:0/2015264434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc009d00 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d37fe700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08c400d9b0 tx=0x7f08c400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4009b40 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c400be80 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d419f960 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c40054b0 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.766+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d419fe90 con 0x7f08d4102340 2024-01-25T16:07:31.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.767+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d40fd850 con 0x7f08d4102340 2024-01-25T16:07:31.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.769+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f08c4005610 con 0x7f08d4102340 2024-01-25T16:07:31.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.769+0000 7f08d0ff9700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 0x7f08bc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:31.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.769+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c40cccb0 con 0x7f08d4102340 2024-01-25T16:07:31.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.769+0000 7f08d2ffd700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 0x7f08bc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:31.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.770+0000 7f08d2ffd700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 0x7f08bc0a3830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:31.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.773+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f08c4096130 con 0x7f08d4102340 2024-01-25T16:07:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.974+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f08d410d0a0 con 0x7f08d4102340 2024-01-25T16:07:31.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.974+0000 7f08d0ff9700 1 -- 172.21.15.33:0/2015264434 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5277 (secure 0 0 0) 0x7f08c4017070 con 0x7f08d4102340 2024-01-25T16:07:31.977 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 88 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-01-25T15:57:54.453261+0000 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-01-25T16:06:49.263669+0000 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 26 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-01-25T16:07:31.978 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-01-25T16:07:31.979 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/229778098,v1:172.21.15.33:6803/229778098] [v2:172.21.15.33:6804/229778098,v1:172.21.15.33:6805/229778098] exists,up 83211537-b4b8-4002-921b-1a85501279cb 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6800/2892864635,v1:172.21.15.76:6801/2892864635] [v2:172.21.15.76:6802/2892864635,v1:172.21.15.76:6803/2892864635] exists,up c90ee0f1-8aa4-48ce-8068-df0d16d4c86f 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6800/1513420968,v1:172.21.15.152:6801/1513420968] [v2:172.21.15.152:6802/1513420968,v1:172.21.15.152:6803/1513420968] exists,up 337d041f-de02-49ec-87c1-d9e211edacff 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/3993087703,v1:172.21.15.33:6811/3993087703] [v2:172.21.15.33:6812/3993087703,v1:172.21.15.33:6813/3993087703] exists,up d47d7676-603d-43dd-84d5-278ad2c00170 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6808/1997772054,v1:172.21.15.76:6809/1997772054] [v2:172.21.15.76:6810/1997772054,v1:172.21.15.76:6811/1997772054] exists,up 83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6808/984769495,v1:172.21.15.152:6809/984769495] [v2:172.21.15.152:6810/984769495,v1:172.21.15.152:6811/984769495] exists,up 722b8a67-2c0c-46f4-bcac-37587042f804 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/2008861778,v1:172.21.15.33:6819/2008861778] [v2:172.21.15.33:6820/2008861778,v1:172.21.15.33:6821/2008861778] exists,up 5c38c5ba-1402-4433-9924-49b144ff520d 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 52 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6816/610247607,v1:172.21.15.76:6817/610247607] [v2:172.21.15.76:6818/610247607,v1:172.21.15.76:6819/610247607] exists,up dc7c7fb6-7719-4f21-8ef5-103d43fa1a49 2024-01-25T16:07:31.980 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6816/281006087,v1:172.21.15.152:6817/281006087] [v2:172.21.15.152:6818/281006087,v1:172.21.15.152:6819/281006087] exists,up aedc05dd-ab90-4ad9-9937-a3a7b56d4f72 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/877151460,v1:172.21.15.33:6827/877151460] [v2:172.21.15.33:6828/877151460,v1:172.21.15.33:6829/877151460] exists,up 57cd9e56-03a7-4be8-98c0-27b71dadb5a7 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 73 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.76:6824/772498776,v1:172.21.15.76:6825/772498776] [v2:172.21.15.76:6826/772498776,v1:172.21.15.76:6827/772498776] exists,up 06adc495-98fd-4870-97cf-0cd51993ca94 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.152:6824/249910241,v1:172.21.15.152:6825/249910241] [v2:172.21.15.152:6826/249910241,v1:172.21.15.152:6827/249910241] exists,up bd65bf47-57d6-49f1-ac41-49ef945e9bf5 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.6 [8,11] 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.13 [2,11] 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.6 [3,6] 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.e [4,8] 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.17 [4,8] 2024-01-25T16:07:31.981 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.20 [5,9] 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.21 [5,9] 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.2c [4,9] 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.3a [4,7] 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/797972698 expires 2024-01-26T15:58:48.018885+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2874813062 expires 2024-01-26T15:58:48.018885+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/1043188550 expires 2024-01-26T15:58:21.567761+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2443786608 expires 2024-01-26T15:58:48.018885+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/1043188550 expires 2024-01-26T15:58:21.567761+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2680727410 expires 2024-01-26T15:58:21.567761+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/3011083041 expires 2024-01-26T15:58:21.567761+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2443786608 expires 2024-01-26T15:58:48.018885+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/123634986 expires 2024-01-26T15:58:48.018885+0000 2024-01-25T16:07:31.982 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/4223658721 expires 2024-01-26T15:58:21.567761+0000 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 msgr2=0x7f08bc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 0x7f08bc0a3830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8008040 comp rx=0 tx=0).stop 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 msgr2=0x7f08d4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f08c400d9b0 tx=0x7f08c400dd70 comp rx=0 tx=0).stop 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 shutdown_connections 2024-01-25T16:07:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f08bc0a1370 0x7f08bc0a3830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08d41075a0 0x7f08d419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4102340 0x7f08d4197820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 --2- 172.21.15.33:0/2015264434 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08d4104c70 0x7f08d4197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 >> 172.21.15.33:0/2015264434 conn(0x7f08d40fbcd0 msgr2=0x7f08d4102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 shutdown_connections 2024-01-25T16:07:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:31.977+0000 7f08d9e9a700 1 -- 172.21.15.33:0/2015264434 wait complete. 2024-01-25T16:07:32.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:32 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2015264434' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-25T16:07:32.608 INFO:teuthology.run_tasks:Running task fs.ready... 2024-01-25T16:07:32.621 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:32.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:32 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2015264434' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-25T16:07:32.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:32 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2015264434' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-01-25T16:07:32.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- 172.21.15.33:0/797392183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8970102340 msgr2=0x7f8970102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:32.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 --2- 172.21.15.33:0/797392183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8970102340 0x7f8970102760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8968009900 tx=0x7f896802bc50 comp rx=0 tx=0).stop 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- 172.21.15.33:0/797392183 shutdown_connections 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 --2- 172.21.15.33:0/797392183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700ff770 0x7f8970109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 --2- 172.21.15.33:0/797392183 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89700fedb0 0x7f89700ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 --2- 172.21.15.33:0/797392183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8970102340 0x7f8970102760 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- 172.21.15.33:0/797392183 >> 172.21.15.33:0/797392183 conn(0x7f89700fa930 msgr2=0x7f89700fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- 172.21.15.33:0/797392183 shutdown_connections 2024-01-25T16:07:32.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- 172.21.15.33:0/797392183 wait complete. 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 Processor -- start 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.725+0000 7f89778ec640 1 -- start start 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700fedb0 0x7f8970101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 0x7f89701005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8970100d20 con 0x7f89700ff770 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8970100e90 con 0x7f89700fedb0 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89701a0c50 con 0x7f8970102340 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44998/0 (socket says 172.21.15.33:44998) 2024-01-25T16:07:32.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 -- 172.21.15.33:0/1279708223 learned_addr learned my addr 172.21.15.33:0/1279708223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8975688640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700fedb0 0x7f8970101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8975e89640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 0x7f89701005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 -- 172.21.15.33:0/1279708223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 msgr2=0x7f89701005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 0x7f89701005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 -- 172.21.15.33:0/1279708223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700fedb0 msgr2=0x7f8970101a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700fedb0 0x7f8970101a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 -- 172.21.15.33:0/1279708223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968009560 con 0x7f89700ff770 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8975e89640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 0x7f89701005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f8974e87640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f896400c950 tx=0x7f896400ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964007c70 con 0x7f89700ff770 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8964007dd0 con 0x7f89700ff770 2024-01-25T16:07:32.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89701a0f00 con 0x7f89700ff770 2024-01-25T16:07:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964018800 con 0x7f89700ff770 2024-01-25T16:07:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.726+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89701a13d0 con 0x7f89700ff770 2024-01-25T16:07:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.727+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8970108200 con 0x7f89700ff770 2024-01-25T16:07:32.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.728+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f896400faa0 con 0x7f89700ff770 2024-01-25T16:07:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.729+0000 7f896e7fc640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 0x7f89580a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.729+0000 7f8975688640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 0x7f89580a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.729+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f89640ccc80 con 0x7f89700ff770 2024-01-25T16:07:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.729+0000 7f8975688640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 0x7f89580a3820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f896802c6d0 tx=0x7f896802c5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.733+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f89640d3080 con 0x7f89700ff770 2024-01-25T16:07:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.912+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f897002cbb0 con 0x7f89700ff770 2024-01-25T16:07:32.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.913+0000 7f896e7fc640 1 -- 172.21.15.33:0/1279708223 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f8964096100 con 0x7f89700ff770 2024-01-25T16:07:32.916 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:32.916 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 msgr2=0x7f89580a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 0x7f89580a3820 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f896802c6d0 tx=0x7f896802c5f0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 msgr2=0x7f8970100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f896400c950 tx=0x7f896400ce10 comp rx=0 tx=0).stop 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 shutdown_connections 2024-01-25T16:07:32.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8970102340 0x7f89701005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f89580a1360 0x7f89580a3820 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89700ff770 0x7f8970100080 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 --2- 172.21.15.33:0/1279708223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89700fedb0 0x7f8970101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 >> 172.21.15.33:0/1279708223 conn(0x7f89700fa930 msgr2=0x7f89701053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 shutdown_connections 2024-01-25T16:07:32.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:32.915+0000 7f89778ec640 1 -- 172.21.15.33:0/1279708223 wait complete. 2024-01-25T16:07:32.930 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:07:32.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:07:33.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.033+0000 7fdafbc61640 1 -- 172.21.15.33:0/225250651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 msgr2=0x7fdaf4100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.033+0000 7fdafbc61640 1 --2- 172.21.15.33:0/225250651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf4100010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009900 tx=0x7fdaec02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- 172.21.15.33:0/225250651 shutdown_connections 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- 172.21.15.33:0/225250651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf4108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- 172.21.15.33:0/225250651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaf4100de0 0x7fdaf4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- 172.21.15.33:0/225250651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf4100010 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- 172.21.15.33:0/225250651 >> 172.21.15.33:0/225250651 conn(0x7fdaf40fb180 msgr2=0x7fdaf40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- 172.21.15.33:0/225250651 shutdown_connections 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- 172.21.15.33:0/225250651 wait complete. 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 Processor -- start 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- start start 2024-01-25T16:07:33.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaf4100de0 0x7fdaf419bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf419e8d0 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf419ea40 con 0x7fdaf41017a0 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdafbc61640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf419ec00 con 0x7fdaf4100de0 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdaf99fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdaf99fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45008/0 (socket says 172.21.15.33:45008) 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.034+0000 7fdaf99fd640 1 -- 172.21.15.33:0/407729948 learned_addr learned my addr 172.21.15.33:0/407729948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdafa1fe640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 -- 172.21.15.33:0/407729948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaf4100de0 msgr2=0x7fdaf419bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaf4100de0 0x7fdaf419bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 -- 172.21.15.33:0/407729948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 msgr2=0x7fdaf419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 -- 172.21.15.33:0/407729948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec009560 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdafa1fe640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf99fd640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0043d0 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaec0054c0 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf419ee20 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec003e60 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.035+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf419f320 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.036+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf4107030 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.037+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fdaec0035f0 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.038+0000 7fdaf2ffd640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 0x7fdadc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.038+0000 7fdaf91fc640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 0x7fdadc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.038+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdaec0ef980 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.038+0000 7fdaf91fc640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 0x7fdadc0a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdae8005fd0 tx=0x7fdae8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.042+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fdaec0b8e00 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.220+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdaf410b690 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.220+0000 7fdaf2ffd640 1 -- 172.21.15.33:0/407729948 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdaf410b690 con 0x7fdaf40ffbf0 2024-01-25T16:07:33.223 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:07:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 msgr2=0x7fdadc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 0x7fdadc0a37a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdae8005fd0 tx=0x7fdae8005850 comp rx=0 tx=0).stop 2024-01-25T16:07:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 msgr2=0x7fdaf40689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009230 tx=0x7fdaec004f40 comp rx=0 tx=0).stop 2024-01-25T16:07:33.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 shutdown_connections 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaf41017a0 0x7fdaf419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaf4100de0 0x7fdaf419bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fdadc0a12e0 0x7fdadc0a37a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 --2- 172.21.15.33:0/407729948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdaf40ffbf0 0x7fdaf40689c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 >> 172.21.15.33:0/407729948 conn(0x7fdaf40fb180 msgr2=0x7fdaf4104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 shutdown_connections 2024-01-25T16:07:33.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.222+0000 7fdafbc61640 1 -- 172.21.15.33:0/407729948 wait complete. 2024-01-25T16:07:33.236 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:07:33.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 -- 172.21.15.33:0/186037008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398100e00 msgr2=0x7f0398101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 --2- 172.21.15.33:0/186037008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398100e00 0x7f0398101280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0380009900 tx=0x7f038002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 -- 172.21.15.33:0/186037008 shutdown_connections 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 --2- 172.21.15.33:0/186037008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f03981017c0 0x7f0398108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 --2- 172.21.15.33:0/186037008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0398100e00 0x7f0398101280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 --2- 172.21.15.33:0/186037008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03980ffc10 0x7f0398100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 -- 172.21.15.33:0/186037008 >> 172.21.15.33:0/186037008 conn(0x7f03980fb1a0 msgr2=0x7f03980fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 -- 172.21.15.33:0/186037008 shutdown_connections 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.339+0000 7f039d083640 1 -- 172.21.15.33:0/186037008 wait complete. 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 Processor -- start 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 -- start start 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 0x7f039819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 0x7f03981a0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981a0a00 con 0x7f03980ffc10 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981a0b70 con 0x7f03981017c0 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039d083640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03981a0d30 con 0x7f0398100e00 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f0396d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f0396d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45016/0 (socket says 172.21.15.33:45016) 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f0396d9d640 1 -- 172.21.15.33:0/3991805599 learned_addr learned my addr 172.21.15.33:0/3991805599 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f039759e640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 0x7f03981a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f0396d9d640 1 -- 172.21.15.33:0/3991805599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 msgr2=0x7f039819ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.340+0000 7f038ffff640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 0x7f039819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0396d9d640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 0x7f039819ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0396d9d640 1 -- 172.21.15.33:0/3991805599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 msgr2=0x7f03981a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0396d9d640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 0x7f03981a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0396d9d640 1 -- 172.21.15.33:0/3991805599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380009560 con 0x7f03980ffc10 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f039759e640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 0x7f03981a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f038ffff640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 0x7f039819ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0396d9d640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0388009c30 con 0x7f03980ffc10 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03981a0fb0 con 0x7f03980ffc10 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0388009d90 con 0x7f03980ffc10 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038800f740 con 0x7f03980ffc10 2024-01-25T16:07:33.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.341+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03981a1430 con 0x7f03980ffc10 2024-01-25T16:07:33.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.342+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0398107000 con 0x7f03980ffc10 2024-01-25T16:07:33.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.343+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f0388010a20 con 0x7f03980ffc10 2024-01-25T16:07:33.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.343+0000 7f0394d99640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 0x7f03840a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.343+0000 7f038ffff640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 0x7f03840a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.344+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f03880cce70 con 0x7f03980ffc10 2024-01-25T16:07:33.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.344+0000 7f038ffff640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 0x7f03840a37d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0380036040 tx=0x7f0380000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.348+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f03880962b0 con 0x7f03980ffc10 2024-01-25T16:07:33.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.529+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f039802cb70 con 0x7f03980ffc10 2024-01-25T16:07:33.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.530+0000 7f0394d99640 1 -- 172.21.15.33:0/3991805599 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03880998d0 con 0x7f03980ffc10 2024-01-25T16:07:33.532 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:07:33.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 msgr2=0x7f03840a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 0x7f03840a37d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0380036040 tx=0x7f0380000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 msgr2=0x7f0398069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f038800d950 tx=0x7f038800de10 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 shutdown_connections 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03981017c0 0x7f03981a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0398100e00 0x7f039819ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f03840a1310 0x7f03840a37d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 --2- 172.21.15.33:0/3991805599 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03980ffc10 0x7f0398069e30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 >> 172.21.15.33:0/3991805599 conn(0x7f03980fb1a0 msgr2=0x7f03981041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 shutdown_connections 2024-01-25T16:07:33.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.532+0000 7f039d083640 1 -- 172.21.15.33:0/3991805599 wait complete. 2024-01-25T16:07:33.544 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:33 smithi033 ceph-mon[31417]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:33.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:33 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1279708223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:33.545 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:33 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/407729948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:33.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:07:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:33 smithi152 ceph-mon[36015]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:33 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1279708223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:33 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/407729948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2592876265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffbf0 msgr2=0x7fd3e4100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2592876265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4100010 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009900 tx=0x7fd3d402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2592876265 shutdown_connections 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2592876265 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 0x7fd3e4108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2592876265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4100de0 0x7fd3e4101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2592876265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4100010 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2592876265 >> 172.21.15.33:0/2592876265 conn(0x7fd3e40fb180 msgr2=0x7fd3e40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2592876265 shutdown_connections 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.647+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2592876265 wait complete. 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 Processor -- start 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 -- start start 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 0x7fd3e419d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419d670 con 0x7fd3e4100de0 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419d7e0 con 0x7fd3e40ffbf0 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3ea4b4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419d9a0 con 0x7fd3e41017a0 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3e37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3e37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45024/0 (socket says 172.21.15.33:45024) 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.648+0000 7fd3e37fe640 1 -- 172.21.15.33:0/2570306592 learned_addr learned my addr 172.21.15.33:0/2570306592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e3fff640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e8a51640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 0x7fd3e419d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 -- 172.21.15.33:0/2570306592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 msgr2=0x7fd3e419d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 0x7fd3e419d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 -- 172.21.15.33:0/2570306592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e40ffbf0 msgr2=0x7fd3e4195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 -- 172.21.15.33:0/2570306592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009560 con 0x7fd3e4100de0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e37fe640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800d950 tx=0x7fd3d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8009c30 con 0x7fd3e4100de0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3d8009d90 con 0x7fd3e4100de0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e419dc80 con 0x7fd3e4100de0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800f740 con 0x7fd3e4100de0 2024-01-25T16:07:33.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.649+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e419e190 con 0x7fd3e4100de0 2024-01-25T16:07:33.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.650+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e4106fb0 con 0x7fd3e4100de0 2024-01-25T16:07:33.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.651+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fd3d8010a20 con 0x7fd3e4100de0 2024-01-25T16:07:33.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.652+0000 7fd3e17fa640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 0x7fd3cc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.652+0000 7fd3e3fff640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 0x7fd3cc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.652+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd3d80cc870 con 0x7fd3e4100de0 2024-01-25T16:07:33.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.652+0000 7fd3e3fff640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 0x7fd3cc0a37a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402c7e0 tx=0x7fd3d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.656+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd3d8095cf0 con 0x7fd3e4100de0 2024-01-25T16:07:33.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:33 smithi076 ceph-mon[35642]: pgmap v340: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 8 op/s 2024-01-25T16:07:33.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:33 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1279708223' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:33.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:33 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/407729948' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.835+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3e410b6a0 con 0x7fd3e4100de0 2024-01-25T16:07:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.836+0000 7fd3e17fa640 1 -- 172.21.15.33:0/2570306592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3d8099310 con 0x7fd3e4100de0 2024-01-25T16:07:33.838 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 msgr2=0x7fd3cc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 0x7fd3cc0a37a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd3d402c7e0 tx=0x7fd3d4000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 msgr2=0x7fd3e41960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800d950 tx=0x7fd3d800de10 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 shutdown_connections 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e41017a0 0x7fd3e419d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fd3cc0a12e0 0x7fd3cc0a37a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e4100de0 0x7fd3e41960c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 --2- 172.21.15.33:0/2570306592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e40ffbf0 0x7fd3e4195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 >> 172.21.15.33:0/2570306592 conn(0x7fd3e40fb180 msgr2=0x7fd3e4104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 shutdown_connections 2024-01-25T16:07:33.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.838+0000 7fd3ea4b4640 1 -- 172.21.15.33:0/2570306592 wait complete. 2024-01-25T16:07:33.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:07:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 -- 172.21.15.33:0/4235163063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb980ffc10 msgr2=0x7fbb98100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/4235163063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb980ffc10 0x7fbb98100030 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009900 tx=0x7fbb8802bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 -- 172.21.15.33:0/4235163063 shutdown_connections 2024-01-25T16:07:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/4235163063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 0x7fbb98108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/4235163063 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb98100e00 0x7fbb98101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/4235163063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb980ffc10 0x7fbb98100030 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 -- 172.21.15.33:0/4235163063 >> 172.21.15.33:0/4235163063 conn(0x7fbb980fb180 msgr2=0x7fbb980fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 -- 172.21.15.33:0/4235163063 shutdown_connections 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.950+0000 7fbb9e15a640 1 -- 172.21.15.33:0/4235163063 wait complete. 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 Processor -- start 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 -- start start 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb980ffc10 0x7fbb98196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 0x7fbb9819e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9819e820 con 0x7fbb98100e00 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9819e990 con 0x7fbb981017c0 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb9e15a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9819eb50 con 0x7fbb980ffc10 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb96ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45032/0 (socket says 172.21.15.33:45032) 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.951+0000 7fbb96ffd640 1 -- 172.21.15.33:0/719836808 learned_addr learned my addr 172.21.15.33:0/719836808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb977fe640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb980ffc10 0x7fbb98196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 -- 172.21.15.33:0/719836808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb980ffc10 msgr2=0x7fbb98196d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb980ffc10 0x7fbb98196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 -- 172.21.15.33:0/719836808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 msgr2=0x7fbb9819e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb97fff640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 0x7fbb9819e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 0x7fbb9819e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 -- 172.21.15.33:0/719836808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb88009560 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb96ffd640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d770 tx=0x7fbb8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c009c30 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb8c00de80 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9819ee30 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c004de0 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.952+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9819f200 con 0x7fbb98100e00 2024-01-25T16:07:33.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.953+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb98106fb0 con 0x7fbb98100e00 2024-01-25T16:07:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.954+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fbb8c004f40 con 0x7fbb98100e00 2024-01-25T16:07:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.955+0000 7fbb94ff9640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 0x7fbb800a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.955+0000 7fbb977fe640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 0x7fbb800a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.955+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbb8c0cc730 con 0x7fbb98100e00 2024-01-25T16:07:33.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.955+0000 7fbb977fe640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 0x7fbb800a37a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb88036040 tx=0x7fbb88005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:33.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:33.959+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbb8c095ae0 con 0x7fbb98100e00 2024-01-25T16:07:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.176+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb9802cc00 con 0x7fbb98100e00 2024-01-25T16:07:34.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.177+0000 7fbb94ff9640 1 -- 172.21.15.33:0/719836808 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7fbb8c097a10 con 0x7fbb98100e00 2024-01-25T16:07:34.180 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:34.180 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:07:34.180 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 msgr2=0x7fbb800a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 0x7fbb800a37a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbb88036040 tx=0x7fbb88005fd0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 msgr2=0x7fbb98197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00d770 tx=0x7fbb8c00dc30 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 shutdown_connections 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb981017c0 0x7fbb9819e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fbb800a12e0 0x7fbb800a37a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb98100e00 0x7fbb98197280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 --2- 172.21.15.33:0/719836808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb980ffc10 0x7fbb98196d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 >> 172.21.15.33:0/719836808 conn(0x7fbb980fb180 msgr2=0x7fbb98104180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 shutdown_connections 2024-01-25T16:07:34.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.179+0000 7fbb9e15a640 1 -- 172.21.15.33:0/719836808 wait complete. 2024-01-25T16:07:34.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 -- 172.21.15.33:0/1126182068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 msgr2=0x7f485c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 --2- 172.21.15.33:0/1126182068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c100030 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f484c009ca0 tx=0x7f484c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 -- 172.21.15.33:0/1126182068 shutdown_connections 2024-01-25T16:07:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 --2- 172.21.15.33:0/1126182068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 0x7f485c108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 --2- 172.21.15.33:0/1126182068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 --2- 172.21.15.33:0/1126182068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c100030 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 -- 172.21.15.33:0/1126182068 >> 172.21.15.33:0/1126182068 conn(0x7f485c0fb180 msgr2=0x7f485c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 -- 172.21.15.33:0/1126182068 shutdown_connections 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 -- 172.21.15.33:0/1126182068 wait complete. 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.293+0000 7f486377d640 1 Processor -- start 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 -- start start 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c19ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 0x7f485c19e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c19e800 con 0x7f485c0ffc10 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c19e970 con 0x7f485c100e00 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f486377d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c19eb30 con 0x7f485c1017c0 2024-01-25T16:07:34.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45054/0 (socket says 172.21.15.33:45054) 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 -- 172.21.15.33:0/3309188390 learned_addr learned my addr 172.21.15.33:0/3309188390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861d1a640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 0x7f485c19e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4860d18640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c19ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 -- 172.21.15.33:0/3309188390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 msgr2=0x7f485c19e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 0x7f485c19e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 -- 172.21.15.33:0/3309188390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 msgr2=0x7f485c19ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c19ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 -- 172.21.15.33:0/3309188390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c009900 con 0x7f485c0ffc10 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4860d18640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f4861519640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f484c037040 tx=0x7f484c032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.294+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c034c70 con 0x7f485c0ffc10 2024-01-25T16:07:34.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.295+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f484c003e30 con 0x7f485c0ffc10 2024-01-25T16:07:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.295+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c19edb0 con 0x7f485c0ffc10 2024-01-25T16:07:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.295+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c03d3f0 con 0x7f485c0ffc10 2024-01-25T16:07:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.295+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c19f200 con 0x7f485c0ffc10 2024-01-25T16:07:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.296+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c106f30 con 0x7f485c0ffc10 2024-01-25T16:07:34.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.297+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f484c03b040 con 0x7f485c0ffc10 2024-01-25T16:07:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.298+0000 7f485a7fc640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 0x7f48440a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.298+0000 7f4860d18640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 0x7f48440a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.298+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f484c0ef580 con 0x7f485c0ffc10 2024-01-25T16:07:34.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.298+0000 7f4860d18640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 0x7f48440a37a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f485c069c60 tx=0x7f4850005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.301+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f484c0b89c0 con 0x7f485c0ffc10 2024-01-25T16:07:34.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.483+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f485c10b6d0 con 0x7f485c0ffc10 2024-01-25T16:07:34.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.484+0000 7f485a7fc640 1 -- 172.21.15.33:0/3309188390 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f484c0bbfe0 con 0x7f485c0ffc10 2024-01-25T16:07:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 msgr2=0x7f48440a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 0x7f48440a37a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f485c069c60 tx=0x7f4850005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 msgr2=0x7f485c069ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f484c037040 tx=0x7f484c032810 comp rx=0 tx=0).stop 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 shutdown_connections 2024-01-25T16:07:34.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f485c1017c0 0x7f485c19e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f485c100e00 0x7f485c19ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f48440a12e0 0x7f48440a37a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.486+0000 7f486377d640 1 --2- 172.21.15.33:0/3309188390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f485c0ffc10 0x7f485c069ef0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.487+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 >> 172.21.15.33:0/3309188390 conn(0x7f485c0fb180 msgr2=0x7f485c104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.487+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 shutdown_connections 2024-01-25T16:07:34.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.487+0000 7f486377d640 1 -- 172.21.15.33:0/3309188390 wait complete. 2024-01-25T16:07:34.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:34 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3991805599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:34 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2570306592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.499 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:34 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/719836808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:34.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:07:34.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 -- 172.21.15.33:0/27875614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad0101f00 msgr2=0x7f6ad0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/27875614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad0101f00 0x7f6ad0102380 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4009900 tx=0x7f6ac402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 -- 172.21.15.33:0/27875614 shutdown_connections 2024-01-25T16:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/27875614 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad01028c0 0x7f6ad0109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/27875614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad0101f00 0x7f6ad0102380 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/27875614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 -- 172.21.15.33:0/27875614 >> 172.21.15.33:0/27875614 conn(0x7f6ad00fc280 msgr2=0x7f6ad00fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 -- 172.21.15.33:0/27875614 shutdown_connections 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.603+0000 7f6ad6dde640 1 -- 172.21.15.33:0/27875614 wait complete. 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 Processor -- start 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 -- start start 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad0101f00 0x7f6ad01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad019e780 con 0x7f6ad01028c0 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad019e8f0 con 0x7f6ad0100d10 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad6dde640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad019eab0 con 0x7f6ad0101f00 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad537b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad537b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45078/0 (socket says 172.21.15.33:45078) 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad537b640 1 -- 172.21.15.33:0/3836788211 learned_addr learned my addr 172.21.15.33:0/3836788211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6acffff640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad0101f00 0x7f6ad01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.604+0000 7f6ad4b7a640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 -- 172.21.15.33:0/3836788211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad0101f00 msgr2=0x7f6ad01971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad0101f00 0x7f6ad01971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 -- 172.21.15.33:0/3836788211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 msgr2=0x7f6ad0196c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0196c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 -- 172.21.15.33:0/3836788211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac4009560 con 0x7f6ad01028c0 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad4b7a640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:34.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad537b640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e970 tx=0x7f6ac000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0009720 con 0x7f6ad01028c0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad019ed90 con 0x7f6ad01028c0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ac00054e0 con 0x7f6ad01028c0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac0010430 con 0x7f6ad01028c0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.605+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad019f260 con 0x7f6ad01028c0 2024-01-25T16:07:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.606+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad0108030 con 0x7f6ad01028c0 2024-01-25T16:07:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.607+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f6ac000ccc0 con 0x7f6ad01028c0 2024-01-25T16:07:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.608+0000 7f6acdffb640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 0x7f6ab80a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.608+0000 7f6ad4b7a640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 0x7f6ab80a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.608+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f6ac0014070 con 0x7f6ad01028c0 2024-01-25T16:07:34.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.608+0000 7f6ad4b7a640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 0x7f6ab80a37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0197b40 tx=0x7f6ac80074a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.612+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6ac00964d0 con 0x7f6ad01028c0 2024-01-25T16:07:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:34 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3991805599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:34 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2570306592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:34 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/719836808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:34 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3991805599' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:34 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2570306592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:34 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/719836808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:34.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.791+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ad002cbb0 con 0x7f6ad01028c0 2024-01-25T16:07:34.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.791+0000 7f6acdffb640 1 -- 172.21.15.33:0/3836788211 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ac0099af0 con 0x7f6ad01028c0 2024-01-25T16:07:34.794 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:07:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 msgr2=0x7f6ab80a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 0x7f6ab80a37a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0197b40 tx=0x7f6ac80074a0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 msgr2=0x7f6ad019e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ac000e970 tx=0x7f6ac000ed70 comp rx=0 tx=0).stop 2024-01-25T16:07:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 shutdown_connections 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f6ab80a12e0 0x7f6ab80a37a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad01028c0 0x7f6ad019e210 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ad0101f00 0x7f6ad01971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 --2- 172.21.15.33:0/3836788211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ad0100d10 0x7f6ad0196c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 >> 172.21.15.33:0/3836788211 conn(0x7f6ad00fc280 msgr2=0x7f6ad0105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 shutdown_connections 2024-01-25T16:07:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.793+0000 7f6ad6dde640 1 -- 172.21.15.33:0/3836788211 wait complete. 2024-01-25T16:07:34.807 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:07:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.909+0000 7fe4879d8640 1 -- 172.21.15.33:0/2636978150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480100d00 msgr2=0x7fe480101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.909+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2636978150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480100d00 0x7fe480101120 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe47c009900 tx=0x7fe47c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.909+0000 7fe4879d8640 1 -- 172.21.15.33:0/2636978150 shutdown_connections 2024-01-25T16:07:34.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.909+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2636978150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe4801028b0 0x7fe480109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2636978150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe480101ef0 0x7fe480102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2636978150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480100d00 0x7fe480101120 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- 172.21.15.33:0/2636978150 >> 172.21.15.33:0/2636978150 conn(0x7fe4800fc290 msgr2=0x7fe4800fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- 172.21.15.33:0/2636978150 shutdown_connections 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- 172.21.15.33:0/2636978150 wait complete. 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 Processor -- start 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- start start 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe480100d00 0x7fe480196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 0x7fe48019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019e7b0 con 0x7fe480101ef0 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019e920 con 0x7fe4801028b0 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe4879d8640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48019eae0 con 0x7fe480100d00 2024-01-25T16:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe484f73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe484f73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45104/0 (socket says 172.21.15.33:45104) 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.910+0000 7fe484f73640 1 -- 172.21.15.33:0/2353554970 learned_addr learned my addr 172.21.15.33:0/2353554970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe485f75640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 0x7fe48019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 -- 172.21.15.33:0/2353554970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe480100d00 msgr2=0x7fe480196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe480100d00 0x7fe480196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 -- 172.21.15.33:0/2353554970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 msgr2=0x7fe48019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 0x7fe48019e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 -- 172.21.15.33:0/2353554970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009560 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe485f75640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 0x7fe48019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe484f73640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe47000d950 tx=0x7fe47000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470009c30 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe470009d90 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48019edc0 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000f740 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.911+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48019f270 con 0x7fe480101ef0 2024-01-25T16:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.912+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4801080c0 con 0x7fe480101ef0 2024-01-25T16:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.913+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe470010a20 con 0x7fe480101ef0 2024-01-25T16:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.914+0000 7fe4767fc640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 0x7fe46c0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.914+0000 7fe485774640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 0x7fe46c0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.914+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4700cc7c0 con 0x7fe480101ef0 2024-01-25T16:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.914+0000 7fe485774640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 0x7fe46c0a37a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe47c036040 tx=0x7fe47c02c5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:34.917+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe470095c40 con 0x7fe480101ef0 2024-01-25T16:07:35.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.096+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe48002cc30 con 0x7fe480101ef0 2024-01-25T16:07:35.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.096+0000 7fe4767fc640 1 -- 172.21.15.33:0/2353554970 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe470020020 con 0x7fe480101ef0 2024-01-25T16:07:35.099 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 msgr2=0x7fe46c0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 0x7fe46c0a37a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe47c036040 tx=0x7fe47c02c5d0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 msgr2=0x7fe4801971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe47000d950 tx=0x7fe47000de10 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 shutdown_connections 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4801028b0 0x7fe48019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe46c0a12e0 0x7fe46c0a37a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe480101ef0 0x7fe4801971e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 --2- 172.21.15.33:0/2353554970 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe480100d00 0x7fe480196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 >> 172.21.15.33:0/2353554970 conn(0x7fe4800fc290 msgr2=0x7fe480105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 shutdown_connections 2024-01-25T16:07:35.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.098+0000 7fe4879d8640 1 -- 172.21.15.33:0/2353554970 wait complete. 2024-01-25T16:07:35.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- 172.21.15.33:0/443426696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 msgr2=0x7f9970102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- 172.21.15.33:0/443426696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f9970102380 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f995c009900 tx=0x7f995c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- 172.21.15.33:0/443426696 shutdown_connections 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- 172.21.15.33:0/443426696 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f99701028c0 0x7f9970109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- 172.21.15.33:0/443426696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f9970102380 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- 172.21.15.33:0/443426696 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9970100d10 0x7f9970101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- 172.21.15.33:0/443426696 >> 172.21.15.33:0/443426696 conn(0x7f99700fc280 msgr2=0x7f99700fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- 172.21.15.33:0/443426696 shutdown_connections 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- 172.21.15.33:0/443426696 wait complete. 2024-01-25T16:07:35.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 Processor -- start 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 -- start start 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 0x7f9970196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.215+0000 7f997720e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99701028c0 0x7f997019e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997019e7b0 con 0x7f9970101f00 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997019e920 con 0x7f99701028c0 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997019eae0 con 0x7f9970100d10 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45108/0 (socket says 172.21.15.33:45108) 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 -- 172.21.15.33:0/1981020480 learned_addr learned my addr 172.21.15.33:0/1981020480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f9974faa640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 0x7f9970196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 -- 172.21.15.33:0/1981020480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 msgr2=0x7f9970196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 0x7f9970196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 -- 172.21.15.33:0/1981020480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99701028c0 msgr2=0x7f997019e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99701028c0 0x7f997019e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 -- 172.21.15.33:0/1981020480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c009560 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f9974faa640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 0x7f9970196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996ffff640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f995c009230 tx=0x7f995c004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c004350 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f995c0044b0 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997019ed60 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c003e10 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.216+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997019f260 con 0x7f9970101f00 2024-01-25T16:07:35.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.217+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970108100 con 0x7f9970101f00 2024-01-25T16:07:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.218+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f995c005000 con 0x7f9970101f00 2024-01-25T16:07:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.219+0000 7f996dffb640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 0x7f99600a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.219+0000 7f9974faa640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 0x7f99600a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.219+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f995c0ef4b0 con 0x7f9970101f00 2024-01-25T16:07:35.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.219+0000 7f9974faa640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 0x7f99600a3820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9970197b70 tx=0x7f9964005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.223+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f995c0b8930 con 0x7f9970101f00 2024-01-25T16:07:35.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.403+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f997002cc30 con 0x7f9970101f00 2024-01-25T16:07:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.403+0000 7f996dffb640 1 -- 172.21.15.33:0/1981020480 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f995c0bbf50 con 0x7f9970101f00 2024-01-25T16:07:35.406 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:07:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 msgr2=0x7f99600a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 0x7f99600a3820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9970197b70 tx=0x7f9964005d50 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 msgr2=0x7f99701971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f995c009230 tx=0x7f995c004ec0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 shutdown_connections 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99701028c0 0x7f997019e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f99600a1360 0x7f99600a3820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9970101f00 0x7f99701971e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 --2- 172.21.15.33:0/1981020480 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9970100d10 0x7f9970196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 >> 172.21.15.33:0/1981020480 conn(0x7f99700fc280 msgr2=0x7f99701052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 shutdown_connections 2024-01-25T16:07:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.406+0000 7f997720e640 1 -- 172.21.15.33:0/1981020480 wait complete. 2024-01-25T16:07:35.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:35 smithi033 ceph-mon[31417]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-01-25T16:07:35.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:35 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3309188390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:35.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:35 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3836788211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.418 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:35 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2353554970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.420 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:07:35.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- 172.21.15.33:0/133768049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c100e00 msgr2=0x7f4e1c101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 --2- 172.21.15.33:0/133768049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c100e00 0x7f4e1c101280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009900 tx=0x7f4e0402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- 172.21.15.33:0/133768049 shutdown_connections 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 --2- 172.21.15.33:0/133768049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c108050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 --2- 172.21.15.33:0/133768049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c100e00 0x7f4e1c101280 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 --2- 172.21.15.33:0/133768049 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c100030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- 172.21.15.33:0/133768049 >> 172.21.15.33:0/133768049 conn(0x7f4e1c0fb180 msgr2=0x7f4e1c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- 172.21.15.33:0/133768049 shutdown_connections 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- 172.21.15.33:0/133768049 wait complete. 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 Processor -- start 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.519+0000 7f4e231a9640 1 -- start start 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e1c100e00 0x7f4e1c199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0a30 con 0x7f4e1c1017c0 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0ba0 con 0x7f4e1c100e00 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1a0d60 con 0x7f4e1c0ffc10 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45132/0 (socket says 172.21.15.33:45132) 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 -- 172.21.15.33:0/3215277000 learned_addr learned my addr 172.21.15.33:0/3215277000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e20f45640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 -- 172.21.15.33:0/3215277000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 msgr2=0x7f4e1c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 -- 172.21.15.33:0/3215277000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e1c100e00 msgr2=0x7f4e1c199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e1c100e00 0x7f4e1c199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 -- 172.21.15.33:0/3215277000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009560 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e20f45640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e21746640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009fd0 tx=0x7f4e0c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c011830 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e0c011e50 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c1a1040 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.520+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00f2e0 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.521+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c1a1550 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.522+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c10b550 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.522+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f4e0c010580 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.523+0000 7f4e19ffb640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 0x7f4e080a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.523+0000 7f4e20f45640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 0x7f4e080a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.523+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4e0c0cc3c0 con 0x7f4e1c1017c0 2024-01-25T16:07:35.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.523+0000 7f4e20f45640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 0x7f4e080a37a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c100c60 tx=0x7f4e14009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.527+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4e0c095840 con 0x7f4e1c1017c0 2024-01-25T16:07:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:35 smithi152 ceph-mon[36015]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-01-25T16:07:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:35 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3309188390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:35 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3836788211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:35 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2353554970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:35 smithi076 ceph-mon[35642]: pgmap v341: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-01-25T16:07:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:35 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3309188390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:35 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3836788211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:35 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2353554970' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:07:35.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.744+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e1c1a17a0 con 0x7f4e1c1017c0 2024-01-25T16:07:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.745+0000 7f4e19ffb640 1 -- 172.21.15.33:0/3215277000 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11340 (secure 0 0 0) 0x7f4e0c098e60 con 0x7f4e1c1017c0 2024-01-25T16:07:35.748 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:35.748 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:05:50.161531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/2179148160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":2179148160},{"type":"v1","addr":"172.21.15.152:6837","nonce":2179148160}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:07:35.748 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 msgr2=0x7f4e080a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 0x7f4e080a37a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c100c60 tx=0x7f4e14009520 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 msgr2=0x7f4e1c1a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c009fd0 tx=0x7f4e0c00b8e0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 shutdown_connections 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f4e080a12e0 0x7f4e080a37a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e1c1017c0 0x7f4e1c1a04f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e1c100e00 0x7f4e1c199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 --2- 172.21.15.33:0/3215277000 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4e1c0ffc10 0x7f4e1c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 >> 172.21.15.33:0/3215277000 conn(0x7f4e1c0fb180 msgr2=0x7f4e1c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 shutdown_connections 2024-01-25T16:07:35.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.747+0000 7f4e231a9640 1 -- 172.21.15.33:0/3215277000 wait complete. 2024-01-25T16:07:35.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:07:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 -- 172.21.15.33:0/1787215718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 msgr2=0x7f3c5c100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/1787215718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c100050 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009900 tx=0x7f3c4c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 -- 172.21.15.33:0/1787215718 shutdown_connections 2024-01-25T16:07:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/1787215718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 0x7f3c5c108070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/1787215718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/1787215718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c100050 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 -- 172.21.15.33:0/1787215718 >> 172.21.15.33:0/1787215718 conn(0x7f3c5c0fb1a0 msgr2=0x7f3c5c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 -- 172.21.15.33:0/1787215718 shutdown_connections 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.860+0000 7f3c61e2d640 1 -- 172.21.15.33:0/1787215718 wait complete. 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 Processor -- start 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 -- start start 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 0x7f3c5c19e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19e830 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19e9a0 con 0x7f3c5c1017e0 2024-01-25T16:07:35.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c61e2d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5c19eb60 con 0x7f3c5c100e20 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45146/0 (socket says 172.21.15.33:45146) 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 -- 172.21.15.33:0/2404664360 learned_addr learned my addr 172.21.15.33:0/2404664360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5bfff640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 0x7f3c5c19e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 -- 172.21.15.33:0/2404664360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 msgr2=0x7f3c5c197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 -- 172.21.15.33:0/2404664360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 msgr2=0x7f3c5c19e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 0x7f3c5c19e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5b7fe640 1 -- 172.21.15.33:0/2404664360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c4c009560 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.861+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c5b7fe640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009230 tx=0x7f3c4c004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0042d0 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5c19ede0 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c4c004430 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c003d10 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.862+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5c19f260 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.863+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5c10b4c0 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.864+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f3c4c005000 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.864+0000 7f3c58ff9640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 0x7f3c440a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.864+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 0x7f3c440a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.864+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f3c4c0ef7e0 con 0x7f3c5c0ffc30 2024-01-25T16:07:35.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.865+0000 7f3c5affd640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 0x7f3c440a37a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c198250 tx=0x7f3c50005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:35.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:35.868+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f3c4c0b8c20 con 0x7f3c5c0ffc30 2024-01-25T16:07:36.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.050+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c5c19f7d0 con 0x7f3c5c0ffc30 2024-01-25T16:07:36.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.052+0000 7f3c58ff9640 1 -- 172.21.15.33:0/2404664360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22448 (secure 0 0 0) 0x7f3c4c0bc240 con 0x7f3c5c0ffc30 2024-01-25T16:07:36.055 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:07:36.055 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":88,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:06:49.263669+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:07:36.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 msgr2=0x7f3c440a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 0x7f3c440a37a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c198250 tx=0x7f3c50005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 msgr2=0x7f3c5c196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c009230 tx=0x7f3c4c004dd0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 shutdown_connections 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c5c1017e0 0x7f3c5c19e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3c5c100e20 0x7f3c5c197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f3c440a12e0 0x7f3c440a37a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 --2- 172.21.15.33:0/2404664360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c5c0ffc30 0x7f3c5c196d40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 >> 172.21.15.33:0/2404664360 conn(0x7f3c5c0fb1a0 msgr2=0x7f3c5c069410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 shutdown_connections 2024-01-25T16:07:36.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.054+0000 7f3c61e2d640 1 -- 172.21.15.33:0/2404664360 wait complete. 2024-01-25T16:07:36.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-01-25T16:05:46.107985+0000', 'modified': '2024-01-25T16:05:50.161531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24536, 'mds_1': 24517, 'mds_2': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.76:6835/3469230010', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3469230010}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3469230010}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24484': {'gid': 24484, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6835/1042830191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 1042830191}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 1042830191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.152:6837/2179148160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6836', 'nonce': 2179148160}, {'type': 'v1', 'addr': '172.21.15.152:6837', 'nonce': 2179148160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24494': {'gid': 24494, 'name': 'h', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.76:6837/2322550881', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2322550881}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2322550881}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24517': {'gid': 24517, 'name': 'i', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.152:6839/2298219777', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6838', 'nonce': 2298219777}, {'type': 'v1', 'addr': '172.21.15.152:6839', 'nonce': 2298219777}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24536': {'gid': 24536, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6839/406718297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6838', 'nonce': 406718297}, {'type': 'v1', 'addr': '172.21.15.76:6839', 'nonce': 406718297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-25T16:07:36.068 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-01-25T16:07:36.068 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-01-25T16:07:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.168+0000 7fcab4f0e640 1 -- 172.21.15.33:0/2318336390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab0101790 msgr2=0x7fcab0103b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.168+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/2318336390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab0101790 0x7fcab0103b80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009900 tx=0x7fcaa002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:36.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- 172.21.15.33:0/2318336390 shutdown_connections 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/2318336390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab01069f0 0x7fcab0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/2318336390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 0x7fcab01064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/2318336390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab0101790 0x7fcab0103b80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- 172.21.15.33:0/2318336390 >> 172.21.15.33:0/2318336390 conn(0x7fcab00fb1a0 msgr2=0x7fcab00fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- 172.21.15.33:0/2318336390 shutdown_connections 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- 172.21.15.33:0/2318336390 wait complete. 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 Processor -- start 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- start start 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab0101790 0x7fcab0196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 0x7fcab0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab019e7d0 con 0x7fcab01069f0 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab019e940 con 0x7fcab0101790 2024-01-25T16:07:36.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.169+0000 7fcab4f0e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab019eb00 con 0x7fcab01040c0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45164/0 (socket says 172.21.15.33:45164) 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 -- 172.21.15.33:0/786767121 learned_addr learned my addr 172.21.15.33:0/786767121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 0x7fcab0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaae59c640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab0101790 0x7fcab0196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 -- 172.21.15.33:0/786767121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 msgr2=0x7fcab0197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 0x7fcab0197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 -- 172.21.15.33:0/786767121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab0101790 msgr2=0x7fcab0196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab0101790 0x7fcab0196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 -- 172.21.15.33:0/786767121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa0009560 con 0x7fcab01069f0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaaed9d640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b970 tx=0x7fcaa800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaa77fe640 1 -- 172.21.15.33:0/786767121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800caa0 con 0x7fcab01069f0 2024-01-25T16:07:36.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaa77fe640 1 -- 172.21.15.33:0/786767121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa800cc00 con 0x7fcab01069f0 2024-01-25T16:07:36.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab019ede0 con 0x7fcab01069f0 2024-01-25T16:07:36.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcaa77fe640 1 -- 172.21.15.33:0/786767121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80075d0 con 0x7fcab01069f0 2024-01-25T16:07:36.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.170+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab019f2e0 con 0x7fcab01069f0 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.172+0000 7fcaa77fe640 1 -- 172.21.15.33:0/786767121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fcaa8014440 con 0x7fcab01069f0 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.172+0000 7fcab7baa740 1 Processor -- start 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 -- start start 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e251ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e259690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e1c0030 con 0x55ad0e252090 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e1bd720 con 0x55ad0e257200 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcab7baa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e1b9010 con 0x55ad0e2516a0 2024-01-25T16:07:36.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaa77fe640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 0x7fca9c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45170/0 (socket says 172.21.15.33:45170) 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 -- 172.21.15.33:0/1425655249 learned_addr learned my addr 172.21.15.33:0/1425655249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaaed9d640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 0x7fca9c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e259690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e251ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaa77fe640 1 -- 172.21.15.33:0/786767121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fcaa80cc2f0 con 0x7fcab01069f0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 -- 172.21.15.33:0/1425655249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 msgr2=0x55ad0e251ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e251ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 -- 172.21.15.33:0/1425655249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 msgr2=0x55ad0e259690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e259690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.173+0000 7fcaae59c640 1 -- 172.21.15.33:0/1425655249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad0e17b680 con 0x55ad0e252090 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcaaed9d640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 0x7fca9c0a3820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8095ea0 tx=0x7fcaa8064a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcaae59c640 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0036040 tx=0x7fcaa002c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a44695299d0701 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcaa4ff9640 1 -- 172.21.15.33:0/1425655249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00048c0 con 0x55ad0e252090 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcaa4ff9640 1 -- 172.21.15.33:0/1425655249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcaa0004500 con 0x55ad0e252090 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcaa4ff9640 1 -- 172.21.15.33:0/1425655249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0003b90 con 0x55ad0e252090 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcab7baa740 1 -- 172.21.15.33:0/1425655249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 msgr2=0x55ad0e256b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0036040 tx=0x7fcaa002c9b0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcab7baa740 1 -- 172.21.15.33:0/1425655249 shutdown_connections 2024-01-25T16:07:36.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e259690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.174+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e256b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1425655249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e251ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- 172.21.15.33:0/1425655249 >> 172.21.15.33:0/1425655249 conn(0x55ad0e24cc10 msgr2=0x55ad0e24d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- 172.21.15.33:0/1425655249 shutdown_connections 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- 172.21.15.33:0/1425655249 wait complete. 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- start start 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 Processor -- start 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e252090 0x55ad0e25bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e262d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e1725f0 con 0x55ad0e2516a0 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e172290 con 0x55ad0e257200 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e137300 con 0x55ad0e252090 2024-01-25T16:07:36.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcab7baa740 10 client.0 _ll_register_callbacks cb 0x55ad0e16d260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45174/0 (socket says 172.21.15.33:45174) 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcaa7fff640 1 -- 172.21.15.33:0/533595308 learned_addr learned my addr 172.21.15.33:0/533595308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcaae59c640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e252090 0x55ad0e25bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.175+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e262d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 -- 172.21.15.33:0/533595308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e252090 msgr2=0x55ad0e25bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e252090 0x55ad0e25bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 -- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 msgr2=0x55ad0e262d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e262d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 -- 172.21.15.33:0/533595308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0109a20 con 0x55ad0e2516a0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca98011d80 tx=0x7fca98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe1dab32236846c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98021430 con 0x55ad0e2516a0 2024-01-25T16:07:36.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca980203d0 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad0e1b1bc0 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca98021590 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 10 client.15228 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ad0e1b7690 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad0e1b1bc0 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca98020800 con 0x55ad0e2516a0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 10 client.15228 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 20 client.15228 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e2689c0 0x55ad0e267400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 4 client.15228 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.176+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 --> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad0deb86c0 con 0x55ad0e2689c0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.177+0000 7fcaae59c640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e2689c0 0x55ad0e267400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.177+0000 7fcaae59c640 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e2689c0 0x55ad0e267400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c4d6f6da66784369 server_cookie=64ff5d405e8a6457 in_seq=0 out_seq=0 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.177+0000 7fca95ffb640 10 client.15228 ms_handle_connect on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fca95ffb640 1 -- 172.21.15.33:0/533595308 <== mds.2 v2:172.21.15.76:6836/2322550881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad0deb86c0 con 0x55ad0e2689c0 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fca95ffb640 10 client.15228 handle_command_reply: tid=0 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 client.15228 shutdown 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 msgr2=0x55ad0e25b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca98011d80 tx=0x7fca98004730 comp rx=0 tx=0).stop 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 shutdown_connections 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e2689c0 0x55ad0e267400 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e257200 0x55ad0e262d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e252090 0x55ad0e25bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 --2- 172.21.15.33:0/533595308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e2516a0 0x55ad0e25b800 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fca95ffb640 0 client.15228 ms_handle_reset on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:36.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 >> 172.21.15.33:0/533595308 conn(0x55ad0e24cc10 msgr2=0x55ad0e24bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 shutdown_connections 2024-01-25T16:07:36.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 1 -- 172.21.15.33:0/533595308 wait complete. 2024-01-25T16:07:36.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.178+0000 7fcab7baa740 20 client.15228 trim_cache size 0 max 16384 2024-01-25T16:07:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 Processor -- start 2024-01-25T16:07:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 -- start start 2024-01-25T16:07:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e262d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e252090 0x55ad0e267940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e25a970 con 0x55ad0e24cc10 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e269ce0 con 0x55ad0e252090 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcab7baa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0deb86c0 con 0x55ad0e2516a0 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45184/0 (socket says 172.21.15.33:45184) 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcaa7fff640 1 -- 172.21.15.33:0/1862926327 learned_addr learned my addr 172.21.15.33:0/1862926327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcaae59c640 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e262d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.194+0000 7fcaa7fff640 1 -- 172.21.15.33:0/1862926327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 msgr2=0x55ad0e262d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e262d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaa7fff640 1 -- 172.21.15.33:0/1862926327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e252090 msgr2=0x55ad0e267940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e252090 0x55ad0e267940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaa7fff640 1 -- 172.21.15.33:0/1862926327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad0e154390 con 0x55ad0e24cc10 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaae59c640 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e262d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca9800a480 tx=0x7fca980046a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2c2b6074fa5cb3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fca97fff640 1 -- 172.21.15.33:0/1862926327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9800a630 con 0x55ad0e24cc10 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fca97fff640 1 -- 172.21.15.33:0/1862926327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca9800ae80 con 0x55ad0e24cc10 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fca97fff640 1 -- 172.21.15.33:0/1862926327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9802b7a0 con 0x55ad0e24cc10 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 -- 172.21.15.33:0/1862926327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 msgr2=0x55ad0e267400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fca9800a480 tx=0x7fca980046a0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 -- 172.21.15.33:0/1862926327 shutdown_connections 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e252090 0x55ad0e267940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e262d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 --2- 172.21.15.33:0/1862926327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e24cc10 0x55ad0e267400 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 -- 172.21.15.33:0/1862926327 >> 172.21.15.33:0/1862926327 conn(0x55ad0e24d360 msgr2=0x55ad0e2534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 -- 172.21.15.33:0/1862926327 shutdown_connections 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.195+0000 7fcab7baa740 1 -- 172.21.15.33:0/1862926327 wait complete. 2024-01-25T16:07:36.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 -- start start 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 Processor -- start 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e24cc10 0x55ad0e260480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e2609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e261ec0 con 0x55ad0e252090 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e262030 con 0x55ad0e24cc10 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad0e2621f0 con 0x55ad0e2516a0 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcab7baa740 10 client.0 _ll_register_callbacks cb 0x7fcaa0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45188/0 (socket says 172.21.15.33:45188) 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 -- 172.21.15.33:0/535169196 learned_addr learned my addr 172.21.15.33:0/535169196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaae59c640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e2609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e24cc10 0x55ad0e260480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 -- 172.21.15.33:0/535169196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 msgr2=0x55ad0e2609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e2609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 -- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e24cc10 msgr2=0x55ad0e260480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e24cc10 0x55ad0e260480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 -- 172.21.15.33:0/535169196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad0e1b7690 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaae59c640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e2609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fcaa7fff640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca980118e0 tx=0x7fca98005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d4bc774bf11018 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.196+0000 7fca95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9802b750 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fca98030020 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad0e1b1ea0 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca980203d0 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 10 client.15234 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ad0e179ee0 con 0x55ad0e252090 2024-01-25T16:07:36.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca98048020 con 0x55ad0e252090 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fca98044590 con 0x55ad0e252090 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 10 client.15234 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 20 client.15234 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e257200 0x55ad0e2bfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 4 client.15234 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 --> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ad0df0bd10 con 0x55ad0e257200 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.197+0000 7fcaae59c640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e257200 0x55ad0e2bfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fca95ffb640 10 client.15234 ms_handle_connect on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcaae59c640 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e257200 0x55ad0e2bfc30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f639c54b3cac7907 server_cookie=95f8aec61b543c9c in_seq=0 out_seq=0 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fca95ffb640 1 -- 172.21.15.33:0/535169196 <== mds.2 v2:172.21.15.76:6836/2322550881 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55ad0df0bd10 con 0x55ad0e257200 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fca95ffb640 10 client.15234 handle_command_reply: tid=0 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 client.15234 shutdown 2024-01-25T16:07:36.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 msgr2=0x55ad0e261980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fca980118e0 tx=0x7fca98005740 comp rx=0 tx=0).stop 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 shutdown_connections 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:6836/2322550881,v1:172.21.15.76:6837/2322550881] conn(0x55ad0e257200 0x55ad0e2bfc30 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad0e252090 0x55ad0e261980 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ad0e2516a0 0x55ad0e2609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 --2- 172.21.15.33:0/535169196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad0e24cc10 0x55ad0e260480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fca95ffb640 0 client.15234 ms_handle_reset on v2:172.21.15.76:6836/2322550881 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 >> 172.21.15.33:0/535169196 conn(0x55ad0e24d360 msgr2=0x55ad0e267400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 shutdown_connections 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 1 -- 172.21.15.33:0/535169196 wait complete. 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.198+0000 7fcab7baa740 20 client.15234 trim_cache size 0 max 16384 2024-01-25T16:07:36.208 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24494, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 106.389483496, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 125.79365455200001 2024-01-25T16:07:36.209 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 msgr2=0x7fca9c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 0x7fca9c0a3820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8095ea0 tx=0x7fcaa8064a30 comp rx=0 tx=0).stop 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 msgr2=0x7fcab019e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b970 tx=0x7fcaa800bd70 comp rx=0 tx=0).stop 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 shutdown_connections 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fca9c0a1360 0x7fca9c0a3820 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcab01069f0 0x7fcab019e260 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcab01040c0 0x7fcab0197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 --2- 172.21.15.33:0/786767121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcab0101790 0x7fcab0196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 >> 172.21.15.33:0/786767121 conn(0x7fcab00fb1a0 msgr2=0x7fcab01020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 shutdown_connections 2024-01-25T16:07:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.199+0000 7fcab4f0e640 1 -- 172.21.15.33:0/786767121 wait complete. 2024-01-25T16:07:36.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-01-25T16:07:36.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/4127726336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 msgr2=0x7f2c141092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/4127726336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c141092b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009ca0 tx=0x7f2c0c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/4127726336 shutdown_connections 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/4127726336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c141092b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/4127726336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c14102060 0x7f2c141024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/4127726336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c140feff0 0x7f2c140ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/4127726336 >> 172.21.15.33:0/4127726336 conn(0x7f2c140fa920 msgr2=0x7f2c140fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/4127726336 shutdown_connections 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/4127726336 wait complete. 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 Processor -- start 2024-01-25T16:07:36.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.313+0000 7f2c1a4fc640 1 -- start start 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140feff0 0x7f2c141019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c14102060 0x7f2c14100010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14100cb0 con 0x7f2c14102a20 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14100e20 con 0x7f2c14102060 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141a0cb0 con 0x7f2c140feff0 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45202/0 (socket says 172.21.15.33:45202) 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 -- 172.21.15.33:0/2923604 learned_addr learned my addr 172.21.15.33:0/2923604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140feff0 0x7f2c141019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 -- 172.21.15.33:0/2923604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140feff0 msgr2=0x7f2c141019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140feff0 0x7f2c141019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 -- 172.21.15.33:0/2923604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c14102060 msgr2=0x7f2c14100010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c14102060 0x7f2c14100010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 -- 172.21.15.33:0/2923604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0c009900 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009430 tx=0x7f2c0c032790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c117fa640 1 -- 172.21.15.33:0/2923604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c034c70 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c117fa640 1 -- 172.21.15.33:0/2923604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c003e30 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c141a0f00 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.314+0000 7f2c117fa640 1 -- 172.21.15.33:0/2923604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c03d3f0 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.315+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141a1380 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.316+0000 7f2c117fa640 1 -- 172.21.15.33:0/2923604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f2c0c03b040 con 0x7f2c14102a20 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 Processor -- start 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 -- start start 2024-01-25T16:07:36.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c117fa640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 0x7f2bfc0a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6ea5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eaab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6e14030 con 0x563cc6eab200 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6e11720 con 0x563cc6ea56a0 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c1d199740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6e0d010 con 0x563cc6ea6090 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 0x7f2bfc0a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45212/0 (socket says 172.21.15.33:45212) 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 -- 172.21.15.33:0/2531218130 learned_addr learned my addr 172.21.15.33:0/2531218130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c117fa640 1 -- 172.21.15.33:0/2923604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c0c0ef2c0 con 0x7f2c14102a20 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6ea5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eaab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 -- 172.21.15.33:0/2531218130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 msgr2=0x563cc6eaab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eaab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 -- 172.21.15.33:0/2531218130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 msgr2=0x563cc6ea5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6ea5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.317+0000 7f2c18a99640 1 -- 172.21.15.33:0/2531218130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cc6dcf680 con 0x563cc6eab200 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eaab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6ea5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 0x7f2bfc0a37a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009460 tx=0x7f2c08009690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0bc0c0 tx=0x7f2c0c0419a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac71691961932bf2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2bfa7fc640 1 -- 172.21.15.33:0/2531218130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0bbd20 con 0x563cc6eab200 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2bfa7fc640 1 -- 172.21.15.33:0/2531218130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c084ff0 con 0x563cc6eab200 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2bfa7fc640 1 -- 172.21.15.33:0/2531218130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0c0510 con 0x563cc6eab200 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 -- 172.21.15.33:0/2531218130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 msgr2=0x563cc6ead690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c0bc0c0 tx=0x7f2c0c0419a0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 -- 172.21.15.33:0/2531218130 shutdown_connections 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ead690 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eaab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2531218130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6ea5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.318+0000 7f2c1d199740 1 -- 172.21.15.33:0/2531218130 >> 172.21.15.33:0/2531218130 conn(0x563cc6ea0c10 msgr2=0x563cc6ea1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- 172.21.15.33:0/2531218130 shutdown_connections 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- 172.21.15.33:0/2531218130 wait complete. 2024-01-25T16:07:36.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- start start 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 Processor -- start 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6eaf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6eab200 0x563cc6eb6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6dc65f0 con 0x563cc6ea6090 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6dc6290 con 0x563cc6ea56a0 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6d8b300 con 0x563cc6eab200 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c1d199740 10 client.0 _ll_register_callbacks cb 0x563cc6dc1260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45228/0 (socket says 172.21.15.33:45228) 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c137fe640 1 -- 172.21.15.33:0/2857183336 learned_addr learned my addr 172.21.15.33:0/2857183336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6eaf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.319+0000 7f2c18a99640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6eab200 0x563cc6eb6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 -- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6eab200 msgr2=0x563cc6eb6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6eab200 0x563cc6eb6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 -- 172.21.15.33:0/2857183336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 msgr2=0x563cc6eaf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6eaf5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 -- 172.21.15.33:0/2857183336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141a1380 con 0x563cc6ea6090 2024-01-25T16:07:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c137fe640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c080127d0 tx=0x7f2c08005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25cb98961bef876d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004220 con 0x563cc6ea6090 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c08004380 con 0x563cc6ea6090 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cc6e05ea0 con 0x563cc6ea6090 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c08005020 con 0x563cc6ea6090 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 10 client.15246 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x563cc6e0b690 con 0x563cc6ea6090 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cc6e05ea0 con 0x563cc6ea6090 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c0801a070 con 0x563cc6ea6090 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 10 client.15246 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 10 client.15246 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 20 client.15246 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6ebc760 0x563cc6ebb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 4 client.15246 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.320+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 --> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cc6e05bc0 con 0x563cc6ebc760 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.321+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6ebc760 0x563cc6ebb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.321+0000 7f2bf37fe640 10 client.15246 ms_handle_connect on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.321+0000 7f2c13fff640 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6ebc760 0x563cc6ebb110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c4ed2098444f887 server_cookie=98c0e68e5dbff8ec in_seq=0 out_seq=0 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2bf37fe640 1 -- 172.21.15.33:0/2857183336 <== mds.1 v2:172.21.15.152:6838/2298219777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cc6e05bc0 con 0x563cc6ebc760 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2bf37fe640 10 client.15246 handle_command_reply: tid=0 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 client.15246 shutdown 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 msgr2=0x563cc6eafb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2c080127d0 tx=0x7f2c08005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 shutdown_connections 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6ebc760 0x563cc6ebb110 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6eab200 0x563cc6eb6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2bf37fe640 0 client.15246 ms_handle_reset on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea6090 0x563cc6eafb00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 --2- 172.21.15.33:0/2857183336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6eaf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 >> 172.21.15.33:0/2857183336 conn(0x563cc6ea0c10 msgr2=0x563cc6e9fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 shutdown_connections 2024-01-25T16:07:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 1 -- 172.21.15.33:0/2857183336 wait complete. 2024-01-25T16:07:36.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.322+0000 7f2c1d199740 20 client.15246 trim_cache size 0 max 16384 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 Processor -- start 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 -- start start 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6e9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6eb7980 con 0x563cc6eab200 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6e14030 con 0x563cc6ea56a0 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c1d199740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6ea6ce0 con 0x563cc6ea6090 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45238/0 (socket says 172.21.15.33:45238) 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 -- 172.21.15.33:0/1888330959 learned_addr learned my addr 172.21.15.33:0/1888330959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c137fe640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c13fff640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6e9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 -- 172.21.15.33:0/1888330959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 msgr2=0x563cc6eb6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 -- 172.21.15.33:0/1888330959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 msgr2=0x563cc6e9bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6e9bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c18a99640 1 -- 172.21.15.33:0/1888330959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cc6da8390 con 0x563cc6eab200 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.338+0000 7f2c13fff640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6e9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c18a99640 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c02c6d0 tx=0x7f2c0c034f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eac8d042a23103 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2bf9ffb640 1 -- 172.21.15.33:0/1888330959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0bcac0 con 0x563cc6eab200 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2bf9ffb640 1 -- 172.21.15.33:0/1888330959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c0c0c4750 con 0x563cc6eab200 2024-01-25T16:07:36.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2bf9ffb640 1 -- 172.21.15.33:0/1888330959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c0041a0 con 0x563cc6eab200 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 -- 172.21.15.33:0/1888330959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 msgr2=0x563cc6ea0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c02c6d0 tx=0x7f2c0c034f60 comp rx=0 tx=0).stop 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 -- 172.21.15.33:0/1888330959 shutdown_connections 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6eab200 0x563cc6ea0ff0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1888330959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6ea56a0 0x563cc6e9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 -- 172.21.15.33:0/1888330959 >> 172.21.15.33:0/1888330959 conn(0x563cc6ea0c10 msgr2=0x563cc6eaf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 -- 172.21.15.33:0/1888330959 shutdown_connections 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.339+0000 7f2c1d199740 1 -- 172.21.15.33:0/1888330959 wait complete. 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 -- start start 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 Processor -- start 2024-01-25T16:07:36.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6eab200 0x563cc6eb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6eaacd0 con 0x563cc6ea56a0 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6eae450 con 0x563cc6eab200 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cc6eb5a20 con 0x563cc6ea6090 2024-01-25T16:07:36.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c1d199740 10 client.0 _ll_register_callbacks cb 0x7f2c08016050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45252/0 (socket says 172.21.15.33:45252) 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 -- 172.21.15.33:0/1949967710 learned_addr learned my addr 172.21.15.33:0/1949967710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c137fe640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c18a99640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6eab200 0x563cc6eb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 -- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 msgr2=0x563cc6eb45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 -- 172.21.15.33:0/1949967710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6eab200 msgr2=0x563cc6eb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6eab200 0x563cc6eb54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.340+0000 7f2c13fff640 1 -- 172.21.15.33:0/1949967710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cc6dcf680 con 0x563cc6ea56a0 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c137fe640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c13fff640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400da40 tx=0x7f2c04010af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feed135f81754fbb server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c04015040 con 0x563cc6ea56a0 2024-01-25T16:07:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2c0400f420 con 0x563cc6ea56a0 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cc6ebc470 con 0x563cc6ea56a0 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c0401b8e0 con 0x563cc6ea56a0 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 10 client.15252 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x563cc6dcdee0 con 0x563cc6ea56a0 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563cc6ebc470 con 0x563cc6ea56a0 2024-01-25T16:07:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2c04024690 con 0x563cc6ea56a0 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 10 client.15252 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 20 client.15252 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6f17440 0x563cc6f138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 4 client.15252 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.341+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 --> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f2c0c084ff0 con 0x563cc6f17440 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c137fe640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6f17440 0x563cc6f138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2bf37fe640 10 client.15252 ms_handle_connect on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c137fe640 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6f17440 0x563cc6f138a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e08144d70b9ba392 server_cookie=96b133fabcd4ddac in_seq=0 out_seq=0 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2bf37fe640 1 -- 172.21.15.33:0/1949967710 <== mds.1 v2:172.21.15.152:6838/2298219777 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x7f2c0c084ff0 con 0x563cc6f17440 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2bf37fe640 10 client.15252 handle_command_reply: tid=0 2024-01-25T16:07:36.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 client.15252 shutdown 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 msgr2=0x563cc6eb40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400da40 tx=0x7f2c04010af0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 shutdown_connections 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777] conn(0x563cc6f17440 0x563cc6f138a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cc6eab200 0x563cc6eb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563cc6ea6090 0x563cc6eb45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2bf37fe640 0 client.15252 ms_handle_reset on v2:172.21.15.152:6838/2298219777 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 --2- 172.21.15.33:0/1949967710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563cc6ea56a0 0x563cc6eb40a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 >> 172.21.15.33:0/1949967710 conn(0x563cc6ea0c10 msgr2=0x563cc6ebb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 shutdown_connections 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 1 -- 172.21.15.33:0/1949967710 wait complete. 2024-01-25T16:07:36.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.342+0000 7f2c1d199740 20 client.15252 trim_cache size 0 max 16384 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24517, 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:36.362 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 107.541982717, 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 122.98183542 2024-01-25T16:07:36.363 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 msgr2=0x7f2bfc0a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 0x7f2bfc0a37a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c08009460 tx=0x7f2c08009690 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 msgr2=0x7f2c14100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009430 tx=0x7f2c0c032790 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 shutdown_connections 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f2bfc0a12e0 0x7f2bfc0a37a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c14102a20 0x7f2c14100550 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c14102060 0x7f2c14100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 --2- 172.21.15.33:0/2923604 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2c140feff0 0x7f2c141019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 >> 172.21.15.33:0/2923604 conn(0x7f2c140fa920 msgr2=0x7f2c141053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 shutdown_connections 2024-01-25T16:07:36.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.343+0000 7f2c1a4fc640 1 -- 172.21.15.33:0/2923604 wait complete. 2024-01-25T16:07:36.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1981020480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:36.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3215277000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.366 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:36 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2404664360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 -- 172.21.15.33:0/2989220223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0100d00 msgr2=0x7f38b0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2989220223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0100d00 0x7f38b0101120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f38a0009900 tx=0x7f38a002bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 -- 172.21.15.33:0/2989220223 shutdown_connections 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2989220223 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b01028b0 0x7f38b0109140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2989220223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0101ef0 0x7f38b0102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2989220223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0100d00 0x7f38b0101120 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 -- 172.21.15.33:0/2989220223 >> 172.21.15.33:0/2989220223 conn(0x7f38b00fc290 msgr2=0x7f38b00fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 -- 172.21.15.33:0/2989220223 shutdown_connections 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 -- 172.21.15.33:0/2989220223 wait complete. 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.467+0000 7f38b5cee640 1 Processor -- start 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 -- start start 2024-01-25T16:07:36.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0100d00 0x7f38b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 0x7f38b00730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0073d80 con 0x7f38b01028b0 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0073ef0 con 0x7f38b0100d00 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38b5cee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b01a0da0 con 0x7f38b0101ef0 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45270/0 (socket says 172.21.15.33:45270) 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 -- 172.21.15.33:0/2530148364 learned_addr learned my addr 172.21.15.33:0/2530148364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 0x7f38b00730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38af7fe640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0100d00 0x7f38b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 -- 172.21.15.33:0/2530148364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 msgr2=0x7f38b00730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 0x7f38b00730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 -- 172.21.15.33:0/2530148364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0100d00 msgr2=0x7f38b0074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0100d00 0x7f38b0074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 -- 172.21.15.33:0/2530148364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009560 con 0x7f38b01028b0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 0x7f38b00730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.468+0000 7f38affff640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38a800db20 tx=0x7f38a800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.469+0000 7f38acff9640 1 -- 172.21.15.33:0/2530148364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a800bea0 con 0x7f38b01028b0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.469+0000 7f38acff9640 1 -- 172.21.15.33:0/2530148364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a8009940 con 0x7f38b01028b0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.469+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b01a0ff0 con 0x7f38b01028b0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.469+0000 7f38acff9640 1 -- 172.21.15.33:0/2530148364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a80106c0 con 0x7f38b01028b0 2024-01-25T16:07:36.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.469+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b01a1480 con 0x7f38b01028b0 2024-01-25T16:07:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38acff9640 1 -- 172.21.15.33:0/2530148364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f38a80041b0 con 0x7f38b01028b0 2024-01-25T16:07:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 Processor -- start 2024-01-25T16:07:36.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 -- start start 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cb3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cb6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5c1d030 con 0x5648b5cae6a0 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5c1a720 con 0x5648b5caf090 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38b898c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5c16010 con 0x5648b5cb4200 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38acff9640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 0x7f38980a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45286/0 (socket says 172.21.15.33:45286) 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.471+0000 7f38aeffd640 1 -- 172.21.15.33:0/2087108185 learned_addr learned my addr 172.21.15.33:0/2087108185 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38affff640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 0x7f38980a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38acff9640 1 -- 172.21.15.33:0/2530148364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38a80cc880 con 0x7f38b01028b0 2024-01-25T16:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38af7fe640 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cb3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 -- 172.21.15.33:0/2087108185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 msgr2=0x5648b5cb6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cb6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 -- 172.21.15.33:0/2087108185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 msgr2=0x5648b5cb3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cb3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 -- 172.21.15.33:0/2087108185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648b5bd8680 con 0x5648b5cae6a0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bdb0 tx=0x7f38a4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6eee8c6a230ac4e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f38affff640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 0x7f38980a37a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f38a800b760 tx=0x7f38a806cae0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f3895ffb640 1 -- 172.21.15.33:0/2087108185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a401a600 con 0x5648b5cae6a0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f3895ffb640 1 -- 172.21.15.33:0/2087108185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a4011e60 con 0x5648b5cae6a0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.472+0000 7f3895ffb640 1 -- 172.21.15.33:0/2087108185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4012ab0 con 0x5648b5cae6a0 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- 172.21.15.33:0/2087108185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 msgr2=0x5648b5caeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f38a400bdb0 tx=0x7f38a4011750 comp rx=0 tx=0).stop 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- 172.21.15.33:0/2087108185 shutdown_connections 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cb6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cb3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 --2- 172.21.15.33:0/2087108185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cae6a0 0x5648b5caeac0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- 172.21.15.33:0/2087108185 >> 172.21.15.33:0/2087108185 conn(0x5648b5ca9c10 msgr2=0x5648b5caa010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- 172.21.15.33:0/2087108185 shutdown_connections 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- 172.21.15.33:0/2087108185 wait complete. 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 -- start start 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.473+0000 7f38b898c740 1 Processor -- start 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5cae6a0 0x5648b5cb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cbfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5bcf5f0 con 0x5648b5caf090 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5bcf290 con 0x5648b5cae6a0 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5b94300 con 0x5648b5cb4200 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 10 client.0 _ll_register_callbacks cb 0x5648b5bca260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45290/0 (socket says 172.21.15.33:45290) 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 -- 172.21.15.33:0/678604942 learned_addr learned my addr 172.21.15.33:0/678604942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 -- 172.21.15.33:0/678604942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 msgr2=0x5648b5cbfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cbfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 -- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5cae6a0 msgr2=0x5648b5cb88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5cae6a0 0x5648b5cb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 -- 172.21.15.33:0/678604942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a400ba90 con 0x5648b5caf090 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38af7fe640 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38a002c240 tx=0x7f38a0004960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b37b45fbf6cf39 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f388effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0004250 con 0x5648b5caf090 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38a0045070 con 0x5648b5caf090 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648b5c0eea0 con 0x5648b5caf090 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a00043b0 con 0x5648b5caf090 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 10 client.15264 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.474+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5648b5c14690 con 0x5648b5caf090 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5648b5c0eea0 con 0x5648b5caf090 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f38a003c040 con 0x5648b5caf090 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 10 client.15264 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 10 client.15264 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 20 client.15264 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5cc5a60 0x5648b5cc4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 4 client.15264 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648b5c0ebc0 con 0x5648b5cc5a60 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38aeffd640 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5cc5a60 0x5648b5cc4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f388effd640 10 client.15264 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.475+0000 7f38aeffd640 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5cc5a60 0x5648b5cc4500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7955ea8789dae65f server_cookie=daba3e8687d8573a in_seq=0 out_seq=0 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f388effd640 1 -- 172.21.15.33:0/678604942 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5648b5c0ebc0 con 0x5648b5cc5a60 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f388effd640 10 client.15264 handle_command_reply: tid=0 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 client.15264 shutdown 2024-01-25T16:07:36.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 msgr2=0x5648b5cb8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f38a002c240 tx=0x7f38a0004960 comp rx=0 tx=0).stop 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 shutdown_connections 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5cc5a60 0x5648b5cc4500 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cb4200 0x5648b5cbfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5caf090 0x5648b5cb8df0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f388effd640 0 client.15264 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 --2- 172.21.15.33:0/678604942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5cae6a0 0x5648b5cb88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 >> 172.21.15.33:0/678604942 conn(0x5648b5ca9c10 msgr2=0x5648b5ca8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 shutdown_connections 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.476+0000 7f38b898c740 1 -- 172.21.15.33:0/678604942 wait complete. 2024-01-25T16:07:36.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.477+0000 7f38b898c740 20 client.15264 trim_cache size 0 max 16384 2024-01-25T16:07:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 Processor -- start 2024-01-25T16:07:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 -- start start 2024-01-25T16:07:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cb0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5cb86f0 con 0x5648b5cb4200 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5caff90 con 0x5648b5caf090 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38b898c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5c0ebf0 con 0x5648b5cae6a0 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45306/0 (socket says 172.21.15.33:45306) 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.492+0000 7f38aeffd640 1 -- 172.21.15.33:0/2811957843 learned_addr learned my addr 172.21.15.33:0/2811957843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38af7fe640 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 -- 172.21.15.33:0/2811957843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 msgr2=0x5648b5cb0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cb0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 -- 172.21.15.33:0/2811957843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 msgr2=0x5648b5cbfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 -- 172.21.15.33:0/2811957843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648b5bb1390 con 0x5648b5cb4200 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38af7fe640 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:36.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38aeffd640 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f38a4011fd0 tx=0x7f38a4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a60a8623634e45 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38957fa640 1 -- 172.21.15.33:0/2811957843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a402b030 con 0x5648b5cb4200 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38957fa640 1 -- 172.21.15.33:0/2811957843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a40044f0 con 0x5648b5cb4200 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38957fa640 1 -- 172.21.15.33:0/2811957843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4012bf0 con 0x5648b5cb4200 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 -- 172.21.15.33:0/2811957843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 msgr2=0x5648b5cb0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f38a4011fd0 tx=0x7f38a4005740 comp rx=0 tx=0).stop 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 -- 172.21.15.33:0/2811957843 shutdown_connections 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cb0ad0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.493+0000 7f38b898c740 1 --2- 172.21.15.33:0/2811957843 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cb0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- 172.21.15.33:0/2811957843 >> 172.21.15.33:0/2811957843 conn(0x5648b5ca9c10 msgr2=0x5648b5ccd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- 172.21.15.33:0/2811957843 shutdown_connections 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- 172.21.15.33:0/2811957843 wait complete. 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- start start 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 Processor -- start 2024-01-25T16:07:36.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5cbf010 con 0x5648b5cb4200 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5cbf180 con 0x5648b5caf090 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648b5cbf340 con 0x5648b5cae6a0 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38b898c740 10 client.0 _ll_register_callbacks cb 0x7f38a400b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45314/0 (socket says 172.21.15.33:45314) 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38aeffd640 1 -- 172.21.15.33:0/1341717358 learned_addr learned my addr 172.21.15.33:0/1341717358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38aeffd640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.494+0000 7f38af7fe640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 -- 172.21.15.33:0/1341717358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 msgr2=0x5648b5cbd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cbd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 -- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 msgr2=0x5648b5cbdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:07:36.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 -- 172.21.15.33:0/1341717358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648b5bd8680 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38aeffd640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b560 tx=0x7f38a4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d492adb698ab8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a4038070 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f38a402c070 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38a4011e60 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38a402b030 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 10 client.15270 fetch_fsmap learned FSMap version 21 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5648b5bd6ee0 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38a4042080 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f38a403b020 con 0x5648b5cb4200 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 10 client.15270 fetch_fsmap finished waiting for FSMap version 21 2024-01-25T16:07:36.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 10 client.15270 resolve_mds: resolved name 'b' to daemon mds.b 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 20 client.15270 populate_metadata read hostname 'smithi033' 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5d20690 0x5648b5d1cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 4 client.15270 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.495+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5648b59156c0 con 0x5648b5d20690 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38af7fe640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5d20690 0x5648b5d1cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f388effd640 10 client.15270 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38af7fe640 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5d20690 0x5648b5d1cb50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9066b27a4b2ea7c server_cookie=7084e16f05c6b2bf in_seq=0 out_seq=0 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f388effd640 1 -- 172.21.15.33:0/1341717358 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5648b59156c0 con 0x5648b5d20690 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f388effd640 10 client.15270 handle_command_reply: tid=0 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 client.15270 shutdown 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 msgr2=0x5648b5cbead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f38a400b560 tx=0x7f38a4005970 comp rx=0 tx=0).stop 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 shutdown_connections 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5648b5d20690 0x5648b5d1cb50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5648b5cb4200 0x5648b5cbead0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5648b5caf090 0x5648b5cbdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f38b898c740 1 --2- 172.21.15.33:0/1341717358 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5648b5cae6a0 0x5648b5cbd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 >> 172.21.15.33:0/1341717358 conn(0x5648b5ca9c10 msgr2=0x5648b5cb0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.496+0000 7f388effd640 0 client.15270 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 shutdown_connections 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b898c740 1 -- 172.21.15.33:0/1341717358 wait complete. 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b898c740 20 client.15270 trim_cache size 0 max 16384 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "55cf2542-bb9a-11ee-95b2-87774f69a715", 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-01-25T16:07:36.506 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24536, 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 109.712186136, 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 21, 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 88, 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 86, 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 114.86094458399999 2024-01-25T16:07:36.507 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 msgr2=0x7f38980a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 0x7f38980a37a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f38a800b760 tx=0x7f38a806cae0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 msgr2=0x7f38b0073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f38a800db20 tx=0x7f38a800b260 comp rx=0 tx=0).stop 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 shutdown_connections 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f38980a12e0 0x7f38980a37a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b01028b0 0x7f38b0073620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38b0101ef0 0x7f38b00730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 --2- 172.21.15.33:0/2530148364 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38b0100d00 0x7f38b0074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 >> 172.21.15.33:0/2530148364 conn(0x7f38b00fc290 msgr2=0x7f38b0105260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 shutdown_connections 2024-01-25T16:07:36.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:36.497+0000 7f38b5cee640 1 -- 172.21.15.33:0/2530148364 wait complete. 2024-01-25T16:07:36.510 INFO:teuthology.run_tasks:Running task kclient... 2024-01-25T16:07:36.537 INFO:tasks.kclient:Mounting kernel clients... 2024-01-25T16:07:36.537 INFO:tasks.kclient:config is {'client.0': {}} 2024-01-25T16:07:36.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:36.537 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-01-25T16:07:36.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:36.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-01-25T16:07:36.621 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-01-25T16:07:36.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:07:36.622 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-01-25T16:07:36.622 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-25T16:07:36.622 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:36.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph fs ls 2024-01-25T16:07:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1981020480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3215277000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:36 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2404664360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1981020480' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:07:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3215277000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:36 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2404664360' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:07:36.904 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:07:37.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:37 smithi033 ceph-mon[31417]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:37 smithi152 ceph-mon[36015]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:37 smithi076 ceph-mon[35642]: pgmap v342: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:38.306 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:38 smithi033 ceph-mon[31417]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:38.634 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:38 smithi152 ceph-mon[36015]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 -- 172.21.15.33:0/171477706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 msgr2=0x7fe32410e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 --2- 172.21.15.33:0/171477706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe32410e630 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe3140099c0 tx=0x7fe314009cd0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 -- 172.21.15.33:0/171477706 shutdown_connections 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 --2- 172.21.15.33:0/171477706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe32410e630 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 --2- 172.21.15.33:0/171477706 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324106650 0x7fe324106a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 --2- 172.21.15.33:0/171477706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324101830 0x7fe324101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 -- 172.21.15.33:0/171477706 >> 172.21.15.33:0/171477706 conn(0x7fe324078910 msgr2=0x7fe324078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 -- 172.21.15.33:0/171477706 shutdown_connections 2024-01-25T16:07:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.662+0000 7fe329775700 1 -- 172.21.15.33:0/171477706 wait complete. 2024-01-25T16:07:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.663+0000 7fe329775700 1 Processor -- start 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.663+0000 7fe329775700 1 -- start start 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.663+0000 7fe329775700 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 0x7fe324073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.663+0000 7fe329775700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.663+0000 7fe329775700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324106650 0x7fe324074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe329775700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3241a6030 con 0x7fe3241021f0 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe329775700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3241a61a0 con 0x7fe324106650 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe329775700 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3241a6360 con 0x7fe324101830 2024-01-25T16:07:38.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60236/0 (socket says 172.21.15.33:60236) 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 -- 172.21.15.33:0/3822917192 learned_addr learned my addr 172.21.15.33:0/3822917192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3237fe700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324106650 0x7fe324074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe322ffd700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 0x7fe324073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 -- 172.21.15.33:0/3822917192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 msgr2=0x7fe324073ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 0x7fe324073ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 -- 172.21.15.33:0/3822917192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324106650 msgr2=0x7fe324074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324106650 0x7fe324074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 -- 172.21.15.33:0/3822917192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3140095f0 con 0x7fe3241021f0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe322ffd700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 0x7fe324073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe3227fc700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c920 tx=0x7fe30c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c007b00 con 0x7fe3241021f0 2024-01-25T16:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3241a6640 con 0x7fe3241021f0 2024-01-25T16:07:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe30c004cf0 con 0x7fe3241021f0 2024-01-25T16:07:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c005520 con 0x7fe3241021f0 2024-01-25T16:07:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.664+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3241a6b40 con 0x7fe3241021f0 2024-01-25T16:07:38.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.665+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324109d10 con 0x7fe3241021f0 2024-01-25T16:07:38.671 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:38 smithi076 ceph-mon[35642]: pgmap v343: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 0 B/s wr, 6 op/s 2024-01-25T16:07:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.667+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fe30c004750 con 0x7fe3241021f0 2024-01-25T16:07:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.668+0000 7fe31bfff700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 0x7fe3100a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.668+0000 7fe322ffd700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 0x7fe3100a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.668+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe30c0ccc20 con 0x7fe3241021f0 2024-01-25T16:07:38.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.668+0000 7fe322ffd700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 0x7fe3100a3830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007eb0 tx=0x7fe31c0073c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:38.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.671+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fe30c0cce60 con 0x7fe3241021f0 2024-01-25T16:07:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.922+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe3240684a0 con 0x7fe3241021f0 2024-01-25T16:07:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.922+0000 7fe31bfff700 1 -- 172.21.15.33:0/3822917192 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe30c01d020 con 0x7fe3241021f0 2024-01-25T16:07:38.926 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-25T16:07:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 msgr2=0x7fe3100a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 0x7fe3100a3830 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe31c007eb0 tx=0x7fe31c0073c0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 msgr2=0x7fe324074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:38.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe30c00c920 tx=0x7fe30c00cce0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 shutdown_connections 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fe3100a1370 0x7fe3100a3830 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe324106650 0x7fe324074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe324101830 0x7fe324073ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 --2- 172.21.15.33:0/3822917192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3241021f0 0x7fe324074020 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.924+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 >> 172.21.15.33:0/3822917192 conn(0x7fe324078910 msgr2=0x7fe3240ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.925+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 shutdown_connections 2024-01-25T16:07:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:38.925+0000 7fe329775700 1 -- 172.21.15.33:0/3822917192 wait complete. 2024-01-25T16:07:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:39 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3822917192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-25T16:07:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:39 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3822917192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-25T16:07:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:39 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3822917192' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-01-25T16:07:39.682 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-25T16:07:39.683 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-25T16:07:39.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:39.683 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-01-25T16:07:39.699 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-01-25T16:07:39.700 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42153sec preferred_lft 42153sec 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-01-25T16:07:39.701 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-25T16:07:39.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-25T16:07:39.702 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-01-25T16:07:39.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:39.863 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-25T16:07:39.890 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-01-25T16:07:39.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:39.891 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-01-25T16:07:39.968 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-01-25T16:07:39.969 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-25T16:07:39.969 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-01-25T16:07:39.969 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-01-25T16:07:39.969 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-25T16:07:39.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:39.969 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-01-25T16:07:39.969 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-01-25T16:07:39.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-01-25T16:07:39.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-01-25T16:07:39.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-01-25T16:07:39.970 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-01-25T16:07:40.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.257 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-01-25T16:07:40.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.273 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-01-25T16:07:40.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.329 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-01-25T16:07:40.329 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-01-25T16:07:40.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-25T16:07:40.329 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-25T16:07:40.329 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-01-25T16:07:40.414 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-25T16:07:40.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-25T16:07:40.415 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-01-25T16:07:40.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.510 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec bash -c ' 2024-01-25T16:07:40.510 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-01-25T16:07:40.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-01-25T16:07:40.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-01-25T16:07:40.511 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-01-25T16:07:40.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-25T16:07:40.553 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:40.611 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-25T16:07:40.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-25T16:07:40.612 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:40.666 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-01-25T16:07:40.666 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-01-25T16:07:40.667 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/960ef3eb-54d7-4234-b24f-2801405b107f 2024-01-25T16:07:40.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/960ef3eb-54d7-4234-b24f-2801405b107f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-25T16:07:40.756 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-25T16:07:40.756 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-01-25T16:07:40.896 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,nofallback 2024-01-25T16:07:40.896 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync". 2024-01-25T16:07:40.896 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.76:3300,172.21.15.152:3300" 2024-01-25T16:07:40.896 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with new device syntax: 0@55cf2542-bb9a-11ee-95b2-87774f69a715.cephfs=/volumes/qa/sv_1/960ef3eb-54d7-4234-b24f-2801405b107f 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,name=0,key=0,mon_addr=172.21.15.33:3300/172.21.15.76:3300/172.21.15.152:3300" will pass to kernel 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: You just mounted a file system that supports labels which does not 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: applications will generate AVC messages and not be allowed access to 2024-01-25T16:07:40.897 INFO:teuthology.orchestra.run.smithi033.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-25T16:07:40.899 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-25T16:07:40.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.899 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:40.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:40.928 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-25T16:07:40.983 INFO:teuthology.orchestra.run.smithi033.stdout:client24905 2024-01-25T16:07:40.984 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T16:07:40.984 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-25T16:07:41.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:41.040 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-25T16:07:41.095 INFO:teuthology.orchestra.run.smithi033.stdout:client24905 2024-01-25T16:07:41.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:41.096 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-25T16:07:41.153 INFO:teuthology.orchestra.run.smithi033.stdout:client24905 2024-01-25T16:07:41.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:07:41.153 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715.client24905/status') 2024-01-25T16:07:41.221 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.24905 (3)172.21.15.33:0/535402423 2024-01-25T16:07:41.221 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-01-25T16:07:41.221 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-01-25T16:07:41.221 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-01-25T16:07:41.221 INFO:teuthology.orchestra.run.smithi033.stderr:70 bytes copied, 2.2303e-05 s, 3.1 MB/s 2024-01-25T16:07:41.222 INFO:teuthology.run_tasks:Running task exec... 2024-01-25T16:07:41.234 INFO:teuthology.task.exec:Executing custom commands... 2024-01-25T16:07:41.234 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:07:41.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-01-25T16:07:41.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 -- 172.21.15.33:0/2954332498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc101f10 msgr2=0x7fa0dc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 --2- 172.21.15.33:0/2954332498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc101f10 0x7fa0dc102390 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009900 tx=0x7fa0c402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 -- 172.21.15.33:0/2954332498 shutdown_connections 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 --2- 172.21.15.33:0/2954332498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc1028d0 0x7fa0dc109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 --2- 172.21.15.33:0/2954332498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc101f10 0x7fa0dc102390 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 --2- 172.21.15.33:0/2954332498 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc100d20 0x7fa0dc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.363+0000 7fa0e202b640 1 -- 172.21.15.33:0/2954332498 >> 172.21.15.33:0/2954332498 conn(0x7fa0dc0fc2b0 msgr2=0x7fa0dc0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- 172.21.15.33:0/2954332498 shutdown_connections 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- 172.21.15.33:0/2954332498 wait complete. 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 Processor -- start 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- start start 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc101f10 0x7fa0dc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:41.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc1028d0 0x7fa0dc19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc19e8a0 con 0x7fa0dc100d20 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc19ea10 con 0x7fa0dc101f10 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.364+0000 7fa0e202b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc19ebd0 con 0x7fa0dc1028d0 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60246/0 (socket says 172.21.15.33:60246) 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 -- 172.21.15.33:0/3101122838 learned_addr learned my addr 172.21.15.33:0/3101122838 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0dbfff640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc1028d0 0x7fa0dc19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 -- 172.21.15.33:0/3101122838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc1028d0 msgr2=0x7fa0dc19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc1028d0 0x7fa0dc19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 -- 172.21.15.33:0/3101122838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc101f10 msgr2=0x7fa0dc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0daffd640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc101f10 0x7fa0dc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc101f10 0x7fa0dc197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 -- 172.21.15.33:0/3101122838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4009560 con 0x7fa0dc100d20 2024-01-25T16:07:41.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0db7fe640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00b9c0 tx=0x7fa0cc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00cba0 con 0x7fa0dc100d20 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0cc00cd00 con 0x7fa0dc100d20 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc19eeb0 con 0x7fa0dc100d20 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0126b0 con 0x7fa0dc100d20 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.365+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc19f300 con 0x7fa0dc100d20 2024-01-25T16:07:41.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.366+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc108040 con 0x7fa0dc100d20 2024-01-25T16:07:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.367+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7fa0cc014990 con 0x7fa0dc100d20 2024-01-25T16:07:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.368+0000 7fa0d8ff9640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 0x7fa0c80a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.368+0000 7fa0daffd640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 0x7fa0c80a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.368+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa0cc0cc590 con 0x7fa0dc100d20 2024-01-25T16:07:41.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.368+0000 7fa0daffd640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 0x7fa0c80a3840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036040 tx=0x7fa0c4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:41.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.372+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa0cc04f600 con 0x7fa0dc100d20 2024-01-25T16:07:41.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:41 smithi152 ceph-mon[36015]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:41 smithi076 ceph-mon[35642]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:41 smithi033 ceph-mon[31417]: pgmap v344: 97 pgs: 97 active+clean; 636 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:07:41.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:41.591+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fa0dc02cc30 con 0x7fa0dc100d20 2024-01-25T16:07:42.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.095+0000 7fa0d8ff9640 1 -- 172.21.15.33:0/3101122838 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa0cc095940 con 0x7fa0dc100d20 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 msgr2=0x7fa0c80a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 0x7fa0c80a3840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4036040 tx=0x7fa0c4000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 msgr2=0x7fa0dc196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00b9c0 tx=0x7fa0cc00be80 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 shutdown_connections 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa0dc1028d0 0x7fa0dc19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0dc101f10 0x7fa0dc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7fa0c80a1380 0x7fa0c80a3840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 --2- 172.21.15.33:0/3101122838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0dc100d20 0x7fa0dc196dc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 >> 172.21.15.33:0/3101122838 conn(0x7fa0dc0fc2b0 msgr2=0x7fa0dc105210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 shutdown_connections 2024-01-25T16:07:42.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.098+0000 7fa0e202b640 1 -- 172.21.15.33:0/3101122838 wait complete. 2024-01-25T16:07:42.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-01-25T16:07:42.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 -- 172.21.15.33:0/1401198482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c1017e0 msgr2=0x7f017c108070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 --2- 172.21.15.33:0/1401198482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c1017e0 0x7f017c108070 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0174009ca0 tx=0x7f017402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 -- 172.21.15.33:0/1401198482 shutdown_connections 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 --2- 172.21.15.33:0/1401198482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c1017e0 0x7f017c108070 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 --2- 172.21.15.33:0/1401198482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c100e20 0x7f017c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 --2- 172.21.15.33:0/1401198482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c0ffc30 0x7f017c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 -- 172.21.15.33:0/1401198482 >> 172.21.15.33:0/1401198482 conn(0x7f017c0fb1a0 msgr2=0x7f017c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 -- 172.21.15.33:0/1401198482 shutdown_connections 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.222+0000 7f0184525640 1 -- 172.21.15.33:0/1401198482 wait complete. 2024-01-25T16:07:42.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 Processor -- start 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 -- start start 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c0ffc30 0x7f017c196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c1017e0 0x7f017c19e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19e6c0 con 0x7f017c100e20 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19e830 con 0x7f017c1017e0 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0184525640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017c19e9f0 con 0x7f017c0ffc30 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0181ac0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0181ac0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60270/0 (socket says 172.21.15.33:60270) 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0181ac0640 1 -- 172.21.15.33:0/4117090231 learned_addr learned my addr 172.21.15.33:0/4117090231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:42.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f01822c1640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c0ffc30 0x7f017c196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0182ac2640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c1017e0 0x7f017c19e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0181ac0640 1 -- 172.21.15.33:0/4117090231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c0ffc30 msgr2=0x7f017c196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.223+0000 7f0181ac0640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c0ffc30 0x7f017c196bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0181ac0640 1 -- 172.21.15.33:0/4117090231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c1017e0 msgr2=0x7f017c19e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0181ac0640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c1017e0 0x7f017c19e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0181ac0640 1 -- 172.21.15.33:0/4117090231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174009900 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0181ac0640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d950 tx=0x7f016c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c009a30 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f016c009b90 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f017c19ecd0 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c0047a0 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.224+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f017c19f190 con 0x7f017c100e20 2024-01-25T16:07:42.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.225+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f017c106f50 con 0x7f017c100e20 2024-01-25T16:07:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.226+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f016c009d00 con 0x7f017c100e20 2024-01-25T16:07:42.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.227+0000 7f01737fe640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 0x7f01680a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.227+0000 7f01822c1640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 0x7f01680a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.227+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f016c0cc5d0 con 0x7f017c100e20 2024-01-25T16:07:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.227+0000 7f01822c1640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 0x7f01680a3840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f017c100c80 tx=0x7f0178005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.231+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f016c095a50 con 0x7f017c100e20 2024-01-25T16:07:42.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-25T16:07:42.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-25T16:07:42.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.436+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f017c02cc20 con 0x7f01680a1380 2024-01-25T16:07:42.444 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-01-25T16:07:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.455+0000 7f01737fe640 1 -- 172.21.15.33:0/4117090231 <== mgr.14152 v2:172.21.15.33:6800/1063041969 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f017c02cc20 con 0x7f01680a1380 2024-01-25T16:07:42.457 INFO:teuthology.orchestra.run.smithi033.stdout:{} 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.456+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 msgr2=0x7f01680a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.456+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 0x7f01680a3840 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f017c100c80 tx=0x7f0178005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.456+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 msgr2=0x7f017c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.456+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f016c00d950 tx=0x7f016c00de10 comp rx=0 tx=0).stop 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 shutdown_connections 2024-01-25T16:07:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f017c1017e0 0x7f017c19e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f01680a1380 0x7f01680a3840 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f017c100e20 0x7f017c197110 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 --2- 172.21.15.33:0/4117090231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f017c0ffc30 0x7f017c196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 >> 172.21.15.33:0/4117090231 conn(0x7f017c0fb1a0 msgr2=0x7f017c104120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 shutdown_connections 2024-01-25T16:07:42.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.457+0000 7f0184525640 1 -- 172.21.15.33:0/4117090231 wait complete. 2024-01-25T16:07:42.470 INFO:teuthology.run_tasks:Running task check-counter... 2024-01-25T16:07:42.481 INFO:teuthology.run_tasks:Running task exec... 2024-01-25T16:07:42.499 INFO:teuthology.task.exec:Executing custom commands... 2024-01-25T16:07:42.500 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:07:42.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-01-25T16:07:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 -- 172.21.15.33:0/266580912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4101840 msgr2=0x7f85e4103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/266580912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4101840 0x7f85e4103c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009900 tx=0x7f85d402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 -- 172.21.15.33:0/266580912 shutdown_connections 2024-01-25T16:07:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/266580912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85e4106aa0 0x7f85e4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/266580912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4104170 0x7f85e4106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/266580912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4101840 0x7f85e4103c30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 -- 172.21.15.33:0/266580912 >> 172.21.15.33:0/266580912 conn(0x7f85e40fb190 msgr2=0x7f85e40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 -- 172.21.15.33:0/266580912 shutdown_connections 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.599+0000 7f85ea9bb640 1 -- 172.21.15.33:0/266580912 wait complete. 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 Processor -- start 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 -- start start 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4101840 0x7f85e4196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85e4104170 0x7f85e4197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419e710 con 0x7f85e4106aa0 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419e880 con 0x7f85e4101840 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85ea9bb640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85e419ea40 con 0x7f85e4104170 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60294/0 (socket says 172.21.15.33:60294) 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 -- 172.21.15.33:0/171662567 learned_addr learned my addr 172.21.15.33:0/171662567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e3fff640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4101840 0x7f85e4196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 -- 172.21.15.33:0/171662567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85e4104170 msgr2=0x7f85e4197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85e4104170 0x7f85e4197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.600+0000 7f85e8f58640 1 -- 172.21.15.33:0/171662567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4101840 msgr2=0x7f85e4196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e8f58640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4101840 0x7f85e4196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e8f58640 1 -- 172.21.15.33:0/171662567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d4009560 con 0x7f85e4106aa0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e8f58640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00eaa0 tx=0x7f85dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00cdb0 con 0x7f85e4106aa0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85dc005020 con 0x7f85e4106aa0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e419ed20 con 0x7f85e4106aa0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0116d0 con 0x7f85e4106aa0 2024-01-25T16:07:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.601+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85e419f1d0 con 0x7f85e4106aa0 2024-01-25T16:07:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.602+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85e40fcc80 con 0x7f85e4106aa0 2024-01-25T16:07:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.603+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 135895+0+0 (secure 0 0 0) 0x7f85dc005190 con 0x7f85e4106aa0 2024-01-25T16:07:42.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.603+0000 7f85e17fa640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 0x7f85d00a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:42.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.603+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85dc0ccea0 con 0x7f85e4106aa0 2024-01-25T16:07:42.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.603+0000 7f85e3fff640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 0x7f85d00a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:42.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.604+0000 7f85e3fff640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 0x7f85d00a3840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85d402c7e0 tx=0x7f85d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.608+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f85dc096290 con 0x7f85e4106aa0 2024-01-25T16:07:42.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:42.440+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:42.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:42.440+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:42.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:42.440+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:42.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:42.440+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:42.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:42.440+0000 7f9538158700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:42.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:42.838+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f85e402cce0 con 0x7f85e4106aa0 2024-01-25T16:07:43.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.104+0000 7f85e17fa640 1 -- 172.21.15.33:0/171662567 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f85dc01f020 con 0x7f85e4106aa0 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 msgr2=0x7f85d00a3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 0x7f85d00a3840 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85d402c7e0 tx=0x7f85d4000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 msgr2=0x7f85e419e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00eaa0 tx=0x7f85dc00ef60 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 shutdown_connections 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f85d00a1380 0x7f85d00a3840 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e4106aa0 0x7f85e419e1a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85e4104170 0x7f85e4197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 --2- 172.21.15.33:0/171662567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85e4101840 0x7f85e4196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 >> 172.21.15.33:0/171662567 conn(0x7f85e40fb190 msgr2=0x7f85e41020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 shutdown_connections 2024-01-25T16:07:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.106+0000 7f85ea9bb640 1 -- 172.21.15.33:0/171662567 wait complete. 2024-01-25T16:07:43.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 -- 172.21.15.33:0/2248975931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64101fa0 msgr2=0x7f7a64102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 --2- 172.21.15.33:0/2248975931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64101fa0 0x7f7a64102420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009900 tx=0x7f7a4c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 -- 172.21.15.33:0/2248975931 shutdown_connections 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 --2- 172.21.15.33:0/2248975931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64102960 0x7f7a641091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 --2- 172.21.15.33:0/2248975931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64101fa0 0x7f7a64102420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 --2- 172.21.15.33:0/2248975931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a640ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 -- 172.21.15.33:0/2248975931 >> 172.21.15.33:0/2248975931 conn(0x7f7a640fa940 msgr2=0x7f7a640fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 -- 172.21.15.33:0/2248975931 shutdown_connections 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.231+0000 7f7a693de640 1 -- 172.21.15.33:0/2248975931 wait complete. 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 Processor -- start 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 -- start start 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a64101910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 0x7f7a640fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64100bf0 con 0x7f7a64102960 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64100d60 con 0x7f7a64101fa0 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a693de640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a641a0cb0 con 0x7f7a640fefd0 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60320/0 (socket says 172.21.15.33:60320) 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a637fe640 1 -- 172.21.15.33:0/1423550948 learned_addr learned my addr 172.21.15.33:0/1423550948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a62ffd640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a64101910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.232+0000 7f7a5a7fc640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 0x7f7a640fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 msgr2=0x7f7a64101910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a64101910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 msgr2=0x7f7a640fff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 0x7f7a640fff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 -- 172.21.15.33:0/1423550948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009560 con 0x7f7a64102960 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a62ffd640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a64101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a5a7fc640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 0x7f7a640fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a637fe640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb40 tx=0x7f7a5c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004b20 con 0x7f7a64102960 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a5c004c80 con 0x7f7a64102960 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a641a0f60 con 0x7f7a64102960 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c010690 con 0x7f7a64102960 2024-01-25T16:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.233+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a641a1390 con 0x7f7a64102960 2024-01-25T16:07:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.234+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a64108170 con 0x7f7a64102960 2024-01-25T16:07:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.235+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f7a5c005020 con 0x7f7a64102960 2024-01-25T16:07:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.236+0000 7f7a60ff9640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 0x7f7a500a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.236+0000 7f7a62ffd640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 msgr2=0x7f7a500a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.236+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7a5c014070 con 0x7f7a64102960 2024-01-25T16:07:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.236+0000 7f7a62ffd640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 0x7f7a500a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T16:07:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.241+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7a5c0966c0 con 0x7f7a64102960 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-mon[36015]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: ignoring --setuser ceph since I am not root 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: ignoring --setgroup ceph since I am not root 2024-01-25T16:07:43.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-mgr[37889]: -- 172.21.15.152:0/2237154742 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557987eeb520 con 0x557987ec0800 2024-01-25T16:07:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-25T16:07:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:07:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-mon[35642]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-25T16:07:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-25T16:07:43.419 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: ignoring --setuser ceph since I am not root 2024-01-25T16:07:43.419 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: ignoring --setgroup ceph since I am not root 2024-01-25T16:07:43.419 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-mgr[37037]: -- 172.21.15.76:0/2665287290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557886c01520 con 0x557886bd6800 2024-01-25T16:07:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.437+0000 7f7a62ffd640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 msgr2=0x7f7a500a3890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.437+0000 7f7a62ffd640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 0x7f7a500a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T16:07:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.438+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7a6402cb10 con 0x7f7a64102960 2024-01-25T16:07:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.442+0000 7f7a60ff9640 1 -- 172.21.15.33:0/1423550948 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a5c099ce0 con 0x7f7a64102960 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 msgr2=0x7f7a500a3890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 0x7f7a500a3890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 msgr2=0x7f7a64100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00eb40 tx=0x7f7a5c00c2d0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 shutdown_connections 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7a500a13d0 0x7f7a500a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a64102960 0x7f7a64100490 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a64101fa0 0x7f7a640fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 --2- 172.21.15.33:0/1423550948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7a640fefd0 0x7f7a64101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 >> 172.21.15.33:0/1423550948 conn(0x7f7a640fa940 msgr2=0x7f7a64105340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 shutdown_connections 2024-01-25T16:07:43.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.444+0000 7f7a693de640 1 -- 172.21.15.33:0/1423550948 wait complete. 2024-01-25T16:07:43.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3101122838' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-01-25T16:07:43.457 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:07:43.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-mon[31417]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2024-01-25T16:07:43.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-01-25T16:07:43.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setuser ceph since I am not root 2024-01-25T16:07:43.458 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: ignoring --setgroup ceph since I am not root 2024-01-25T16:07:43.459 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-01-25T16:07:43.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- 172.21.15.33:0/3941274990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c100900 msgr2=0x7f078c100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 --2- 172.21.15.33:0/3941274990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c100900 0x7f078c100d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0784009ca0 tx=0x7f078402c180 comp rx=0 tx=0).stop 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- 172.21.15.33:0/3941274990 shutdown_connections 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 --2- 172.21.15.33:0/3941274990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c100900 0x7f078c100d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 --2- 172.21.15.33:0/3941274990 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c0fff60 0x7f078c1003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 --2- 172.21.15.33:0/3941274990 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 0x7f078c0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- 172.21.15.33:0/3941274990 >> 172.21.15.33:0/3941274990 conn(0x7f078c0fb180 msgr2=0x7f078c0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- 172.21.15.33:0/3941274990 shutdown_connections 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- 172.21.15.33:0/3941274990 wait complete. 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 Processor -- start 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.561+0000 7f07945f5640 1 -- start start 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 0x7f078c198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c100900 0x7f078c1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1a08c0 con 0x7f078c0fff60 2024-01-25T16:07:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1a0a30 con 0x7f078c0ff600 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1a0bf0 con 0x7f078c100900 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60342/0 (socket says 172.21.15.33:60342) 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 -- 172.21.15.33:0/883389720 learned_addr learned my addr 172.21.15.33:0/883389720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0792b92640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c100900 0x7f078c1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0792391640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 0x7f078c198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 -- 172.21.15.33:0/883389720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c100900 msgr2=0x7f078c1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c100900 0x7f078c1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 -- 172.21.15.33:0/883389720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 msgr2=0x7f078c198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 0x7f078c198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 -- 172.21.15.33:0/883389720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009900 con 0x7f078c0fff60 2024-01-25T16:07:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f0791b90640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e9f0 tx=0x7f077c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c009700 con 0x7f078c0fff60 2024-01-25T16:07:43.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f077c00cd80 con 0x7f078c0fff60 2024-01-25T16:07:43.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c1a0ed0 con 0x7f078c0fff60 2024-01-25T16:07:43.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.562+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c018860 con 0x7f078c0fff60 2024-01-25T16:07:43.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.563+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c1a13a0 con 0x7f078c0fff60 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.564+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c10a350 con 0x7f078c0fff60 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.565+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f077c0189c0 con 0x7f078c0fff60 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.565+0000 7f07837fe640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 0x7f07780a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.565+0000 7f0792391640 1 -- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 msgr2=0x7f07780a3910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.565+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f077c014070 con 0x7f078c0fff60 2024-01-25T16:07:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.565+0000 7f0792391640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 0x7f07780a3910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T16:07:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.569+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f077c052600 con 0x7f078c0fff60 2024-01-25T16:07:43.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.750+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f078c02cb50 con 0x7f078c0fff60 2024-01-25T16:07:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.755+0000 7f07837fe640 1 -- 172.21.15.33:0/883389720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7f077c0962b0 con 0x7f078c0fff60 2024-01-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 msgr2=0x7f07780a3910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 0x7f07780a3910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 msgr2=0x7f078c199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f077c00e9f0 tx=0x7f077c00eeb0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 shutdown_connections 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f078c100900 0x7f078c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f07780a1450 0x7f07780a3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f078c0fff60 0x7f078c199310 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 --2- 172.21.15.33:0/883389720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f078c0ff600 0x7f078c198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 >> 172.21.15.33:0/883389720 conn(0x7f078c0fb180 msgr2=0x7f078c0fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 shutdown_connections 2024-01-25T16:07:43.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.757+0000 7f07945f5640 1 -- 172.21.15.33:0/883389720 wait complete. 2024-01-25T16:07:43.769 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:43.694+0000 7efd93e23200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T16:07:43.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-01-25T16:07:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 -- 172.21.15.33:0/1006341204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d54100c30 msgr2=0x7f7d541010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 --2- 172.21.15.33:0/1006341204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d54100c30 0x7f7d541010b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7d44009900 tx=0x7f7d4402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 -- 172.21.15.33:0/1006341204 shutdown_connections 2024-01-25T16:07:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 --2- 172.21.15.33:0/1006341204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d541015f0 0x7f7d54107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 --2- 172.21.15.33:0/1006341204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d54100c30 0x7f7d541010b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 --2- 172.21.15.33:0/1006341204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 0x7f7d540ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 -- 172.21.15.33:0/1006341204 >> 172.21.15.33:0/1006341204 conn(0x7f7d540fafd0 msgr2=0x7f7d540fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 -- 172.21.15.33:0/1006341204 shutdown_connections 2024-01-25T16:07:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.874+0000 7f7d53fff640 1 -- 172.21.15.33:0/1006341204 wait complete. 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.875+0000 7f7d53fff640 1 Processor -- start 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.875+0000 7f7d53fff640 1 -- start start 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 0x7f7d54195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d54100c30 0x7f7d54195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 0x7f7d5419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d4a0 con 0x7f7d541015f0 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d610 con 0x7f7d540ffa40 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d53fff640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5419d7d0 con 0x7f7d54100c30 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 0x7f7d5419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 0x7f7d5419cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60356/0 (socket says 172.21.15.33:60356) 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 -- 172.21.15.33:0/2427739802 learned_addr learned my addr 172.21.15.33:0/2427739802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 0x7f7d54195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d527fc640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d54100c30 0x7f7d54195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d54100c30 msgr2=0x7f7d54195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d54100c30 0x7f7d54195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 msgr2=0x7f7d54195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 0x7f7d54195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d44009560 con 0x7f7d541015f0 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d540ffa40 0x7f7d54195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.876+0000 7f7d537fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 0x7f7d5419cf60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00eb10 tx=0x7f7d4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.877+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004b20 con 0x7f7d541015f0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.877+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d5419dab0 con 0x7f7d541015f0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.877+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d4c004c80 con 0x7f7d541015f0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.877+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c010690 con 0x7f7d541015f0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.877+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5419df90 con 0x7f7d541015f0 2024-01-25T16:07:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.878+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d54106e50 con 0x7f7d541015f0 2024-01-25T16:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.879+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f7d4c005020 con 0x7f7d541015f0 2024-01-25T16:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.879+0000 7f7d589fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.879+0000 7f7d52ffd640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.879+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f7d4c014070 con 0x7f7d541015f0 2024-01-25T16:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.879+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-25T16:07:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:43.883+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7d4c096590 con 0x7f7d541015f0 2024-01-25T16:07:43.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:43.658+0000 7f025f4eb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T16:07:43.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:43.751+0000 7f025f4eb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T16:07:44.064 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:43.794+0000 7efd93e23200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T16:07:44.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:44.080+0000 7f7d52ffd640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:44.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:44.080+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-25T16:07:44.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:44.089+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d5402cb10 con 0x7f7d3c0a1320 2024-01-25T16:07:44.105 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:43.654+0000 7f2691638200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-25T16:07:44.105 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:43.745+0000 7f2691638200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-25T16:07:44.377 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-25T16:07:44.377 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-mon[36015]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-01-25T16:07:44.377 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1423550948' entity='client.admin' 2024-01-25T16:07:44.378 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/883389720' entity='client.admin' 2024-01-25T16:07:44.388 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-25T16:07:44.388 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-mon[35642]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-01-25T16:07:44.388 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1423550948' entity='client.admin' 2024-01-25T16:07:44.388 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/883389720' entity='client.admin' 2024-01-25T16:07:44.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/171662567' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-01-25T16:07:44.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-mon[31417]: mgrmap e19: x(active, since 8m), standbys: y, z 2024-01-25T16:07:44.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1423550948' entity='client.admin' 2024-01-25T16:07:44.458 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/883389720' entity='client.admin' 2024-01-25T16:07:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:44.481+0000 7f7d52ffd640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:44.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:44.481+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-25T16:07:44.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.374+0000 7f2691638200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T16:07:44.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.465+0000 7f2691638200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T16:07:44.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.550+0000 7f2691638200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T16:07:44.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.386+0000 7f025f4eb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T16:07:44.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.478+0000 7f025f4eb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T16:07:44.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.563+0000 7f025f4eb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T16:07:44.793 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:44.455+0000 7efd93e23200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-25T16:07:44.793 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:44.549+0000 7efd93e23200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-25T16:07:44.793 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:44.636+0000 7efd93e23200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-25T16:07:44.966 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.702+0000 7f2691638200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T16:07:44.966 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.785+0000 7f2691638200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T16:07:44.977 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.714+0000 7f025f4eb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T16:07:44.977 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.798+0000 7f025f4eb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T16:07:45.059 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:44.790+0000 7efd93e23200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-25T16:07:45.059 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:44.875+0000 7efd93e23200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-25T16:07:45.237 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:44.962+0000 7f2691638200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T16:07:45.237 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:45.148+0000 7f2691638200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:45.246 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:44.975+0000 7f025f4eb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T16:07:45.246 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:45.157+0000 7f025f4eb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:45.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:45.282+0000 7f7d52ffd640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:45.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:45.282+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-25T16:07:45.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:45.056+0000 7efd93e23200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-25T16:07:45.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:45.243+0000 7efd93e23200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:45.632 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:45.234+0000 7f2691638200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T16:07:45.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:45.328+0000 7f2691638200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T16:07:45.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:45.416+0000 7f2691638200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T16:07:45.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:45.244+0000 7f025f4eb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T16:07:45.668 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:45.338+0000 7f025f4eb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T16:07:45.669 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:45.427+0000 7f025f4eb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T16:07:45.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:45.332+0000 7efd93e23200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-25T16:07:45.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:45.429+0000 7efd93e23200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-25T16:07:45.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:45.519+0000 7efd93e23200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-25T16:07:46.793 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:46.468+0000 7f2691638200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T16:07:46.793 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:46.598+0000 7f2691638200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T16:07:46.810 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:46.480+0000 7f025f4eb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T16:07:46.810 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:46.611+0000 7f025f4eb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T16:07:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:46.883+0000 7f7d52ffd640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/1063041969 2024-01-25T16:07:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:46.883+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-01-25T16:07:46.940 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:46.599+0000 7efd93e23200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-25T16:07:46.940 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:46.734+0000 7efd93e23200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-25T16:07:47.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:46.790+0000 7f2691638200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T16:07:47.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:47.025+0000 7f2691638200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T16:07:47.168 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:46.808+0000 7f025f4eb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T16:07:47.168 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:47.037+0000 7f025f4eb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T16:07:47.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:46.937+0000 7efd93e23200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-25T16:07:47.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:47.170+0000 7efd93e23200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-25T16:07:47.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:47.205+0000 7f2691638200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:47.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:47.388+0000 7f2691638200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T16:07:47.664 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:47.214+0000 7f025f4eb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:47.664 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:47.397+0000 7f025f4eb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T16:07:47.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:47.356+0000 7efd93e23200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:47.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:47.548+0000 7efd93e23200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-25T16:07:47.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:47.662+0000 7f025f4eb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:47.918 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:47.749+0000 7f025f4eb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:48.132 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:47.654+0000 7f2691638200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:48.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:47.741+0000 7f2691638200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:48.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:47.826+0000 7efd93e23200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-25T16:07:48.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:47.916+0000 7efd93e23200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-25T16:07:48.418 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:48.142+0000 7f025f4eb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T16:07:48.633 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:48.138+0000 7f2691638200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T16:07:48.814 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:48.318+0000 7efd93e23200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-25T16:07:49.383 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:48.929+0000 7f2691638200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T16:07:49.418 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:48.932+0000 7f025f4eb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T16:07:49.564 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:49.143+0000 7efd93e23200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-25T16:07:49.793 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:49.498+0000 7f025f4eb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T16:07:49.793 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:49.595+0000 7f025f4eb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T16:07:49.793 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y[37033]: 2024-01-25T16:07:49.791+0000 7f025f4eb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T16:07:49.802 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:49.503+0000 7f2691638200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T16:07:49.802 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:49.600+0000 7f2691638200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T16:07:49.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:49.864+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 20) v1 ==== 135912+0+0 (secure 0 0 0) 0x7f7d4c096960 con 0x7f7d541015f0 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:49.727+0000 7efd93e23200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:49.830+0000 7efd93e23200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: Standby manager daemon y restarted 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: Standby manager daemon y started 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:07:50.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: Standby manager daemon z restarted 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: Standby manager daemon z started 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:07:50.047 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:49 smithi033 ceph-mon[31417]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:50.068+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mgrmap(e 21) v1 ==== 135580+0+0 (secure 0 0 0) 0x7f7d4c0cac90 con 0x7f7d541015f0 2024-01-25T16:07:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:50.068+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 msgr2=0x7f7d3c0a37e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-01-25T16:07:50.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:50.068+0000 7f7d589fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/1063041969,v1:172.21.15.33:6801/1063041969] conn(0x7f7d3c0a1320 0x7f7d3c0a37e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z[37885]: 2024-01-25T16:07:49.799+0000 7f2691638200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: Standby manager daemon y restarted 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: Standby manager daemon y started 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: Standby manager daemon z restarted 2024-01-25T16:07:50.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: Standby manager daemon z started 2024-01-25T16:07:50.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:07:50.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:07:50.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:49 smithi152 ceph-mon[36015]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: Standby manager daemon y restarted 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: Standby manager daemon y started 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.76:0/1984924405' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: Standby manager daemon z restarted 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: Standby manager daemon z started 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-01-25T16:07:50.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:49 smithi076 ceph-mon[35642]: from='mgr.? 172.21.15.152:0/681606782' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-01-25T16:07:50.314 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.043+0000 7efd93e23200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-25T16:07:50.734 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf2e59700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf2e59700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf2e59700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf2e59700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf2e59700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf6e61700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf6e61700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf6e61700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf6e61700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:50.735 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:07:50.332+0000 7efcf6e61700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: Active manager daemon x restarted 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: Activating manager daemon x 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: osdmap e89: 12 total, 12 up, 12 in 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: mgrmap e21: x(active, starting, since 0.0133933s), standbys: y, z 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:07:51.065 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:07:51.066 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:07:51.067 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: Manager daemon x is now available 2024-01-25T16:07:51.068 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T16:07:51.069 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:50 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T16:07:51.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.105+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mon.0 v2:172.21.15.33:3300/0 9 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7f7d4c054e00 con 0x7f7d541015f0 2024-01-25T16:07:51.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.106+0000 7f7d589fe640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7d3c0310a0 0x7f7d3c031480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.106+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d5402cb10 con 0x7f7d3c0310a0 2024-01-25T16:07:51.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.106+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7d3c0310a0 0x7f7d3c031480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.107+0000 7f7d52ffd640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7d3c0310a0 0x7f7d3c031480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d54100a90 tx=0x7f7d48005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:51.133 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: Active manager daemon x restarted 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: Activating manager daemon x 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: osdmap e89: 12 total, 12 up, 12 in 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: mgrmap e21: x(active, starting, since 0.0133933s), standbys: y, z 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:07:51.134 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:07:51.135 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: Manager daemon x is now available 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.136 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.137 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T16:07:51.137 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:50 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T16:07:51.168 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: mgrmap e20: x(active, since 9m), standbys: y, z 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: Active manager daemon x restarted 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: Activating manager daemon x 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: osdmap e89: 12 total, 12 up, 12 in 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: mgrmap e21: x(active, starting, since 0.0133933s), standbys: y, z 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-01-25T16:07:51.169 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-01-25T16:07:51.170 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: Manager daemon x is now available 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-01-25T16:07:51.171 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:50 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-01-25T16:07:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.173+0000 7f7d589fe640 1 -- 172.21.15.33:0/2427739802 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7d5402cb10 con 0x7f7d3c0310a0 2024-01-25T16:07:51.176 INFO:teuthology.orchestra.run.smithi033.stdout:Schedule set for path / 2024-01-25T16:07:51.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7d3c0310a0 msgr2=0x7f7d3c031480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7d3c0310a0 0x7f7d3c031480 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d54100a90 tx=0x7f7d48005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 msgr2=0x7f7d5419cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 --2- 172.21.15.33:0/2427739802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d541015f0 0x7f7d5419cf60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00eb10 tx=0x7f7d4c00c2d0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d537fe640 1 -- 172.21.15.33:0/2427739802 reap_dead start 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 shutdown_connections 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 >> 172.21.15.33:0/2427739802 conn(0x7f7d540fafd0 msgr2=0x7f7d54104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 shutdown_connections 2024-01-25T16:07:51.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.176+0000 7f7d53fff640 1 -- 172.21.15.33:0/2427739802 wait complete. 2024-01-25T16:07:51.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 -- 172.21.15.33:0/79910566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08104160 msgr2=0x7f4b08106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 --2- 172.21.15.33:0/79910566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08104160 0x7f4b08106550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009900 tx=0x7f4af802bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 -- 172.21.15.33:0/79910566 shutdown_connections 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 --2- 172.21.15.33:0/79910566 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08106a90 0x7f4b08108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 --2- 172.21.15.33:0/79910566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08104160 0x7f4b08106550 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 --2- 172.21.15.33:0/79910566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08101830 0x7f4b08103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 -- 172.21.15.33:0/79910566 >> 172.21.15.33:0/79910566 conn(0x7f4b080fb1c0 msgr2=0x7f4b080fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 -- 172.21.15.33:0/79910566 shutdown_connections 2024-01-25T16:07:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.302+0000 7f4b106af640 1 -- 172.21.15.33:0/79910566 wait complete. 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 Processor -- start 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 -- start start 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 0x7f4b08196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0819e740 con 0x7f4b08106a90 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0819e8b0 con 0x7f4b08104160 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b106af640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0819ea70 con 0x7f4b08101830 2024-01-25T16:07:51.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0ec4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0dc4a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0e44b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 0x7f4b08196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0ec4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57382/0 (socket says 172.21.15.33:57382) 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0ec4c640 1 -- 172.21.15.33:0/839114124 learned_addr learned my addr 172.21.15.33:0/839114124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.303+0000 7f4b0dc4a640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:43914/0 (socket says 172.21.15.33:43914) 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 -- 172.21.15.33:0/839114124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 msgr2=0x7f4b08196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 0x7f4b08196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 -- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 msgr2=0x7f4b0819e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 -- 172.21.15.33:0/839114124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8009560 con 0x7f4b08104160 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0e44b640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 0x7f4b08196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0ec4c640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b0dc4a640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4af802c170 tx=0x7f4af8004e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8004350 con 0x7f4b08104160 2024-01-25T16:07:51.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af80044b0 con 0x7f4b08104160 2024-01-25T16:07:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b0819ecf0 con 0x7f4b08104160 2024-01-25T16:07:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8003e50 con 0x7f4b08104160 2024-01-25T16:07:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.304+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0819f1d0 con 0x7f4b08104160 2024-01-25T16:07:51.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.305+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b080fccb0 con 0x7f4b08104160 2024-01-25T16:07:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.307+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7f4af80035f0 con 0x7f4b08104160 2024-01-25T16:07:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.308+0000 7f4aff7fe640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 0x7f4af40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.308+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4af80ef280 con 0x7f4b08104160 2024-01-25T16:07:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.308+0000 7f4b0e44b640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 0x7f4af40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.308+0000 7f4b0e44b640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 0x7f4af40a3650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b08198130 tx=0x7f4b04005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:51.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.311+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mon.2 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4af80b8850 con 0x7f4b08104160 2024-01-25T16:07:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.524+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b0802cb80 con 0x7f4af40a1190 2024-01-25T16:07:51.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.584+0000 7f4aff7fe640 1 -- 172.21.15.33:0/839114124 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f4b0802cb80 con 0x7f4af40a1190 2024-01-25T16:07:51.587 INFO:teuthology.orchestra.run.smithi033.stdout:Retention added to path / 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 msgr2=0x7f4af40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 0x7f4af40a3650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4b08198130 tx=0x7f4b04005f90 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 msgr2=0x7f4b081971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4af802c170 tx=0x7f4af8004e50 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 shutdown_connections 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4af40a1190 0x7f4af40a3650 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b08106a90 0x7f4b0819e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4b08104160 0x7f4b081971a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 --2- 172.21.15.33:0/839114124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b08101830 0x7f4b08196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 >> 172.21.15.33:0/839114124 conn(0x7f4b080fb1c0 msgr2=0x7f4b081022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:51.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 shutdown_connections 2024-01-25T16:07:51.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.586+0000 7f4b106af640 1 -- 172.21.15.33:0/839114124 wait complete. 2024-01-25T16:07:51.599 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-01-25T16:07:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 -- 172.21.15.33:0/2624172287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc8102a50 msgr2=0x7febc81092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 --2- 172.21.15.33:0/2624172287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc8102a50 0x7febc81092e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7febc0009ca0 tx=0x7febc002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 -- 172.21.15.33:0/2624172287 shutdown_connections 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 --2- 172.21.15.33:0/2624172287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc8102a50 0x7febc81092e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 --2- 172.21.15.33:0/2624172287 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102090 0x7febc8102510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 --2- 172.21.15.33:0/2624172287 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc80ff020 0x7febc80ff440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 -- 172.21.15.33:0/2624172287 >> 172.21.15.33:0/2624172287 conn(0x7febc80fa950 msgr2=0x7febc80fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 -- 172.21.15.33:0/2624172287 shutdown_connections 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.702+0000 7febd07f5640 1 -- 172.21.15.33:0/2624172287 wait complete. 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 Processor -- start 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 -- start start 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc8102090 0x7febc8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102a50 0x7febc819e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e6b0 con 0x7febc80ff020 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e820 con 0x7febc8102090 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febd07f5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc819e9e0 con 0x7febc8102a50 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57400/0 (socket says 172.21.15.33:57400) 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 -- 172.21.15.33:0/3989721794 learned_addr learned my addr 172.21.15.33:0/3989721794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febced92640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102a50 0x7febc819e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febcdd90640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc8102090 0x7febc8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 -- 172.21.15.33:0/3989721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102a50 msgr2=0x7febc819e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102a50 0x7febc819e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 -- 172.21.15.33:0/3989721794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc8102090 msgr2=0x7febc8197100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc8102090 0x7febc8197100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.703+0000 7febce591640 1 -- 172.21.15.33:0/3989721794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009900 con 0x7febc80ff020 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febce591640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febc400e9c0 tx=0x7febc400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:51.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4009700 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febc40044e0 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc819ecc0 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4010430 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.704+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc819f1d0 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.705+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8108160 con 0x7febc80ff020 2024-01-25T16:07:51.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.706+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7febc4003f60 con 0x7febc80ff020 2024-01-25T16:07:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.706+0000 7febbf7fe640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 0x7febb40a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.706+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7febc4014070 con 0x7febc80ff020 2024-01-25T16:07:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.706+0000 7febcdd90640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 0x7febb40a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.707+0000 7febcdd90640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 0x7febb40a36d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7febc81980d0 tx=0x7febb800b390 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.711+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7febc4096250 con 0x7febc80ff020 2024-01-25T16:07:51.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.947+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7febc802cc30 con 0x7febb40a1210 2024-01-25T16:07:51.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.956+0000 7febbf7fe640 1 -- 172.21.15.33:0/3989721794 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7febc802cc30 con 0x7febb40a1210 2024-01-25T16:07:51.958 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-01-25T00:00:00", "created": "2024-01-25T16:07:51", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-01-25T16:07:51.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 msgr2=0x7febb40a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 0x7febb40a36d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7febc81980d0 tx=0x7febb800b390 comp rx=0 tx=0).stop 2024-01-25T16:07:51.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 msgr2=0x7febc8196bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7febc400e9c0 tx=0x7febc400ee80 comp rx=0 tx=0).stop 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 shutdown_connections 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7febb40a1210 0x7febb40a36d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7febc8102a50 0x7febc819e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febc8102090 0x7febc8197100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 --2- 172.21.15.33:0/3989721794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7febc80ff020 0x7febc8196bc0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 >> 172.21.15.33:0/3989721794 conn(0x7febc80fa950 msgr2=0x7febc8105330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 shutdown_connections 2024-01-25T16:07:51.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:51.958+0000 7febd07f5640 1 -- 172.21.15.33:0/3989721794 wait complete. 2024-01-25T16:07:51.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-01-25T16:07:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 -- 172.21.15.33:0/983793618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414100d50 msgr2=0x7ff414101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 --2- 172.21.15.33:0/983793618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414100d50 0x7ff414101170 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff404009900 tx=0x7ff40402bc30 comp rx=0 tx=0).stop 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 -- 172.21.15.33:0/983793618 shutdown_connections 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 --2- 172.21.15.33:0/983793618 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414102900 0x7ff414109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 --2- 172.21.15.33:0/983793618 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4141023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 --2- 172.21.15.33:0/983793618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414100d50 0x7ff414101170 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 -- 172.21.15.33:0/983793618 >> 172.21.15.33:0/983793618 conn(0x7ff4140fc2c0 msgr2=0x7ff4140fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.085+0000 7ff41c05d640 1 -- 172.21.15.33:0/983793618 shutdown_connections 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 -- 172.21.15.33:0/983793618 wait complete. 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 Processor -- start 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 -- start start 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 0x7ff414074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4140730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff414073d80 con 0x7ff414102900 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff414073ef0 con 0x7ff414100d50 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41c05d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4141a0cc0 con 0x7ff414101f40 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41a5fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41a5fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57422/0 (socket says 172.21.15.33:57422) 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff41a5fa640 1 -- 172.21.15.33:0/4132361617 learned_addr learned my addr 172.21.15.33:0/4132361617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff419df9640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 0x7ff414074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.086+0000 7ff4195f8640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4140730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 -- 172.21.15.33:0/4132361617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 msgr2=0x7ff4140730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4140730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 -- 172.21.15.33:0/4132361617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 msgr2=0x7ff414074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:52.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 0x7ff414074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 -- 172.21.15.33:0/4132361617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009490 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff419df9640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 0x7ff414074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff4195f8640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4140730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41a5fa640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00eb40 tx=0x7ff40c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004b20 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff40c004c80 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404009560 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c010690 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.087+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4141a1210 con 0x7ff414102900 2024-01-25T16:07:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.088+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff414108070 con 0x7ff414102900 2024-01-25T16:07:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.089+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 22) v1 ==== 135707+0+0 (secure 0 0 0) 0x7ff40c005020 con 0x7ff414102900 2024-01-25T16:07:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.090+0000 7ff40affd640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 0x7ff4000a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:07:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.090+0000 7ff419df9640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 0x7ff4000a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:07:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.090+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff40c014070 con 0x7ff414102900 2024-01-25T16:07:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.090+0000 7ff419df9640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 0x7ff4000a3650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff40402c170 tx=0x7ff404000bc0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:07:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.094+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7ff40c096600 con 0x7ff414102900 2024-01-25T16:07:52.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.307+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff41402cc20 con 0x7ff4000a1190 2024-01-25T16:07:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:52 smithi033 ceph-mon[31417]: mgrmap e22: x(active, since 1.04973s), standbys: y, z 2024-01-25T16:07:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:52 smithi033 ceph-mon[31417]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:52 smithi033 ceph-mon[31417]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:52 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:52 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.314+0000 7ff40affd640 1 -- 172.21.15.33:0/4132361617 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7ff41402cc20 con 0x7ff4000a1190 2024-01-25T16:07:52.316 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-01-25T16:07:52.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 msgr2=0x7ff4000a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 0x7ff4000a3650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff40402c170 tx=0x7ff404000bc0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 msgr2=0x7ff414073620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00eb40 tx=0x7ff40c00c2d0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 shutdown_connections 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff4000a1190 0x7ff4000a3650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff414102900 0x7ff414073620 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff414101f40 0x7ff4140730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 --2- 172.21.15.33:0/4132361617 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff414100d50 0x7ff414074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 >> 172.21.15.33:0/4132361617 conn(0x7ff4140fc2c0 msgr2=0x7ff414105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 shutdown_connections 2024-01-25T16:07:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:07:52.316+0000 7ff41c05d640 1 -- 172.21.15.33:0/4132361617 wait complete. 2024-01-25T16:07:52.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-01-25T16:07:52.358 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-01-25T16:07:52.369 INFO:teuthology.run_tasks:Running task background_exec... 2024-01-25T16:07:52.396 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:07:52.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-01-25T16:07:52.397 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:52 smithi152 ceph-mon[36015]: mgrmap e22: x(active, since 1.04973s), standbys: y, z 2024-01-25T16:07:52.397 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:52 smithi152 ceph-mon[36015]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:52.397 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:52 smithi152 ceph-mon[36015]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:52.397 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:52 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.398 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:52 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.400 INFO:teuthology.run_tasks:Running task workunit... 2024-01-25T16:07:52.414 INFO:tasks.workunit:Pulling workunits from ref 661946e7044723130c03627323f6a879a9848cb5 2024-01-25T16:07:52.414 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-25T16:07:52.414 INFO:tasks.workunit:timeout=3h 2024-01-25T16:07:52.414 INFO:tasks.workunit:cleanup=True 2024-01-25T16:07:52.415 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:52 smithi076 ceph-mon[35642]: mgrmap e22: x(active, since 1.04973s), standbys: y, z 2024-01-25T16:07:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:52 smithi076 ceph-mon[35642]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:52 smithi076 ceph-mon[35642]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:52 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:52 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-01-25 16:05:52.475711197 +0000 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-01-25 16:05:52.475711197 +0000 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-01-25 16:07:40.922494986 +0000 2024-01-25T16:07:52.430 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-01-25 16:05:52.475711197 +0000 2024-01-25T16:07:52.431 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-01-25T16:07:52.431 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-01-25T16:07:52.505 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 661946e7044723130c03627323f6a879a9848cb5 2024-01-25T16:07:52.560 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: [25/Jan/2024:16:07:51] ENGINE Bus STARTING 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: [25/Jan/2024:16:07:51] ENGINE Serving on https://172.21.15.33:7150 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: [25/Jan/2024:16:07:51] ENGINE Serving on http://172.21.15.33:8765 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: [25/Jan/2024:16:07:51] ENGINE Bus STARTED 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:53 smithi152 ceph-mon[36015]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-25T16:07:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: [25/Jan/2024:16:07:51] ENGINE Bus STARTING 2024-01-25T16:07:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: [25/Jan/2024:16:07:51] ENGINE Serving on https://172.21.15.33:7150 2024-01-25T16:07:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: [25/Jan/2024:16:07:51] ENGINE Serving on http://172.21.15.33:8765 2024-01-25T16:07:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: [25/Jan/2024:16:07:51] ENGINE Bus STARTED 2024-01-25T16:07:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:53 smithi076 ceph-mon[35642]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-25T16:07:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: [25/Jan/2024:16:07:51] ENGINE Bus STARTING 2024-01-25T16:07:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: [25/Jan/2024:16:07:51] ENGINE Serving on https://172.21.15.33:7150 2024-01-25T16:07:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: [25/Jan/2024:16:07:51] ENGINE Serving on http://172.21.15.33:8765 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: [25/Jan/2024:16:07:51] ENGINE Bus STARTED 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:53.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:53 smithi033 ceph-mon[31417]: mgrmap e23: x(active, since 3s), standbys: y, z 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:54.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:54 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:54.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:54 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:07:54.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:54.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:07:54.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:54.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:54 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:07:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:07:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:07:55.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi033:/etc/ceph/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi076:/etc/ceph/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi152:/etc/ceph/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.384 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:55.385 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi033:/etc/ceph/ceph.conf 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi076:/etc/ceph/ceph.conf 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi152:/etc/ceph/ceph.conf 2024-01-25T16:07:55.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:55.421 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi033:/etc/ceph/ceph.conf 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi076:/etc/ceph/ceph.conf 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi152:/etc/ceph/ceph.conf 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.conf 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi076:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi152:/etc/ceph/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: Updating smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/config/ceph.client.admin.keyring 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.566 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:55.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:07:55.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:07:55.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:07:55.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:07:57.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:57 smithi152 ceph-mon[36015]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:57.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:57 smithi076 ceph-mon[35642]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:57 smithi033 ceph-mon[31417]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-01-25T16:07:59.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:07:59 smithi152 ceph-mon[36015]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-25T16:07:59.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:07:59 smithi076 ceph-mon[35642]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-25T16:07:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:07:59 smithi033 ceph-mon[31417]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-01-25T16:08:01.064 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:08:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:08:00.595+0000 7efcdfaf3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:08:01.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:08:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:08:00.595+0000 7efcdfaf3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:08:01.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:08:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:08:00.595+0000 7efcdfaf3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:08:01.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:08:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:08:00.595+0000 7efcdfaf3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:08:01.065 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:08:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:08:00.595+0000 7efcdfaf3700 -1 client.0 error registering admin socket command: (17) File exists 2024-01-25T16:08:01.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:01 smithi152 ceph-mon[36015]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-25T16:08:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:01 smithi076 ceph-mon[35642]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-25T16:08:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:01 smithi033 ceph-mon[31417]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-01-25T16:08:03.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:03 smithi152 ceph-mon[36015]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-01-25T16:08:03.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:03 smithi076 ceph-mon[35642]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-01-25T16:08:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:03 smithi033 ceph-mon[31417]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 109 KiB/s wr, 68 op/s 2024-01-25T16:08:04.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:04 smithi152 ceph-mon[36015]: mgrmap e25: x(active, since 13s), standbys: y, z 2024-01-25T16:08:04.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:04 smithi076 ceph-mon[35642]: mgrmap e25: x(active, since 13s), standbys: y, z 2024-01-25T16:08:04.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:04 smithi033 ceph-mon[31417]: mgrmap e25: x(active, since 13s), standbys: y, z 2024-01-25T16:08:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:05 smithi076 ceph-mon[35642]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:05.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 -- 172.21.15.33:0/4214251518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8100f10 msgr2=0x7fc9f81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 --2- 172.21.15.33:0/4214251518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8100f10 0x7fc9f81012f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009930 tx=0x7fc9f402bc30 comp rx=0 tx=0).stop 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 -- 172.21.15.33:0/4214251518 shutdown_connections 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 --2- 172.21.15.33:0/4214251518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8105fe0 0x7fc9f810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 --2- 172.21.15.33:0/4214251518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f81018c0 0x7fc9f8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 --2- 172.21.15.33:0/4214251518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8100f10 0x7fc9f81012f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 -- 172.21.15.33:0/4214251518 >> 172.21.15.33:0/4214251518 conn(0x7fc9f80fc7b0 msgr2=0x7fc9f80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 -- 172.21.15.33:0/4214251518 shutdown_connections 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.539+0000 7fc9ffead640 1 -- 172.21.15.33:0/4214251518 wait complete. 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 Processor -- start 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 -- start start 2024-01-25T16:08:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8100f10 0x7fc9f8072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f8073a70 0x7fc9f81a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81a59a0 con 0x7fc9f8105fe0 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81a5b10 con 0x7fc9f8100f10 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9ffead640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f81a5cd0 con 0x7fc9f8073a70 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60366/0 (socket says 172.21.15.33:60366) 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fdc49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8100f10 0x7fc9f8072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 -- 172.21.15.33:0/1578868726 learned_addr learned my addr 172.21.15.33:0/1578868726 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f8073a70 0x7fc9f81a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 -- 172.21.15.33:0/1578868726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f8073a70 msgr2=0x7fc9f81a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f8073a70 0x7fc9f81a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 -- 172.21.15.33:0/1578868726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8100f10 msgr2=0x7fc9f8072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8100f10 0x7fc9f8072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.540+0000 7fc9fd448640 1 -- 172.21.15.33:0/1578868726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4009560 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800d740 tx=0x7fc9e800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9eeffd640 1 -- 172.21.15.33:0/1578868726 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8009c30 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9eeffd640 1 -- 172.21.15.33:0/1578868726 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e800de50 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f81a5fb0 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9eeffd640 1 -- 172.21.15.33:0/1578868726 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8004ea0 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.541+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f81a63e0 con 0x7fc9f8105fe0 2024-01-25T16:08:05.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.543+0000 7fc9eeffd640 1 -- 172.21.15.33:0/1578868726 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7fc9e8005020 con 0x7fc9f8105fe0 2024-01-25T16:08:05.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 Processor -- start 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 -- start start 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9eeffd640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 0x7fc9e40a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da710470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 0x5632da719240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 0x5632da71bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da67f070 con 0x5632da719780 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da67c760 con 0x5632da7109b0 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fca02a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da678050 con 0x5632da710090 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 0x7fc9e40a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9eeffd640 1 -- 172.21.15.33:0/1578868726 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc9e80ccb70 con 0x7fc9f8105fe0 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fe44a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 0x5632da71bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fdc49640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da710470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 0x5632da719240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fe44a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 0x5632da71bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60376/0 (socket says 172.21.15.33:60376) 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fdc49640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da710470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:54224/0 (socket says 172.21.15.33:54224) 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.544+0000 7fc9fe44a640 1 -- 172.21.15.33:0/4237123189 learned_addr learned my addr 172.21.15.33:0/4237123189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 -- 172.21.15.33:0/4237123189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 msgr2=0x5632da710470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da710470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 -- 172.21.15.33:0/4237123189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 msgr2=0x5632da71bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 0x5632da71bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 -- 172.21.15.33:0/4237123189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632da66fe70 con 0x5632da7109b0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 0x7fc9e40a3b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009dc0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9fd448640 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 0x5632da719240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80ca150 tx=0x7fc9e80995c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=481072eb3f90c972 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9e3fff640 1 -- 172.21.15.33:0/4237123189 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80625e0 con 0x5632da7109b0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9e3fff640 1 -- 172.21.15.33:0/4237123189 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e80623c0 con 0x5632da7109b0 2024-01-25T16:08:05.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.545+0000 7fc9e3fff640 1 -- 172.21.15.33:0/4237123189 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80978e0 con 0x5632da7109b0 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- 172.21.15.33:0/4237123189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 msgr2=0x5632da719240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 0x5632da719240 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80ca150 tx=0x7fc9e80995c0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- 172.21.15.33:0/4237123189 shutdown_connections 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da719780 0x5632da71bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da7109b0 0x5632da719240 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 --2- 172.21.15.33:0/4237123189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da710470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- 172.21.15.33:0/4237123189 >> 172.21.15.33:0/4237123189 conn(0x5632da70a9b0 msgr2=0x5632da70ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- 172.21.15.33:0/4237123189 shutdown_connections 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- 172.21.15.33:0/4237123189 wait complete. 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 -- start start 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.546+0000 7fca02a71740 1 Processor -- start 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da7171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da6311e0 con 0x5632da7109b0 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da630e80 con 0x5632da710090 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da5f60c0 con 0x5632da719780 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fca02a71740 10 client.0 _ll_register_callbacks cb 0x5632da61bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fdc49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fdc49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:41876/0 (socket says 172.21.15.33:41876) 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60382/0 (socket says 172.21.15.33:60382) 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fdc49640 1 -- 172.21.15.33:0/675827099 learned_addr learned my addr 172.21.15.33:0/675827099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da7171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 -- 172.21.15.33:0/675827099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 msgr2=0x5632da7171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da7171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 -- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 msgr2=0x5632da716770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fd448640 1 -- 172.21.15.33:0/675827099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f81a63e0 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.547+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9fd448640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8099200 tx=0x7fc9e8063580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cede79f5a4c5467 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8099e30 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc9e8099730 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632da671040 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e809d880 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 10 client.15414 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5632da6766d0 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5632da671040 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc9e8062f30 con 0x5632da7109b0 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 10 client.15414 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:05.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 20 client.15414 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7291f0 0x5632da72b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 4 client.15414 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.548+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632da670d60 con 0x5632da7291f0 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.549+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7291f0 0x5632da72b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.549+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7291f0 0x5632da72b5e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad8a8f74117eba5 server_cookie=c4566476766507a3 in_seq=0 out_seq=0 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.549+0000 7fc9e17fa640 10 client.15414 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fc9e17fa640 1 -- 172.21.15.33:0/675827099 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632da670d60 con 0x5632da7291f0 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fc9e17fa640 10 client.15414 handle_command_reply: tid=0 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 client.15414 shutdown 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 msgr2=0x5632da716cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8099200 tx=0x7fc9e8063580 comp rx=0 tx=0).stop 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 shutdown_connections 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7291f0 0x5632da72b5e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da7171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da716cb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fc9e17fa640 0 client.15414 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 --2- 172.21.15.33:0/675827099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da716770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 >> 172.21.15.33:0/675827099 conn(0x5632da70a9b0 msgr2=0x5632da70c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 shutdown_connections 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 1 -- 172.21.15.33:0/675827099 wait complete. 2024-01-25T16:08:05.561 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.550+0000 7fca02a71740 20 client.15414 trim_cache size 0 max 16384 2024-01-25T16:08:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:05 smithi033 ceph-mon[31417]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 Processor -- start 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 -- start start 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da718ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da713a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da725ff0 con 0x5632da7109b0 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da70a4e0 con 0x5632da710090 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fca02a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da728ff0 con 0x5632da719780 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60388/0 (socket says 172.21.15.33:60388) 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 -- 172.21.15.33:0/2860397535 learned_addr learned my addr 172.21.15.33:0/2860397535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da713a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da718ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 -- 172.21.15.33:0/2860397535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 msgr2=0x5632da713a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da713a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 -- 172.21.15.33:0/2860397535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 msgr2=0x5632da718ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da718ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 -- 172.21.15.33:0/2860397535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632da613060 con 0x5632da7109b0 2024-01-25T16:08:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da713a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da718ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9fd448640 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80961a0 tx=0x7fc9e8097ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a35f056ed95c3c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.564+0000 7fc9e37fe640 1 -- 172.21.15.33:0/2860397535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8099e30 con 0x5632da7109b0 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fc9e37fe640 1 -- 172.21.15.33:0/2860397535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e8099730 con 0x5632da7109b0 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fc9e37fe640 1 -- 172.21.15.33:0/2860397535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e80a1770 con 0x5632da7109b0 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- 172.21.15.33:0/2860397535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 msgr2=0x5632da717010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80961a0 tx=0x7fc9e8097ce0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- 172.21.15.33:0/2860397535 shutdown_connections 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da719780 0x5632da713a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da717010 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 --2- 172.21.15.33:0/2860397535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da710090 0x5632da718ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- 172.21.15.33:0/2860397535 >> 172.21.15.33:0/2860397535 conn(0x5632da70a9b0 msgr2=0x5632da715060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- 172.21.15.33:0/2860397535 shutdown_connections 2024-01-25T16:08:05.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- 172.21.15.33:0/2860397535 wait complete. 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.565+0000 7fca02a71740 1 -- start start 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 Processor -- start 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da7205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 0x5632da71ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da71f490 con 0x5632da7109b0 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da71f600 con 0x5632da719780 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632da71f7c0 con 0x5632da710090 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fca02a71740 10 client.0 _ll_register_callbacks cb 0x7fc9e8004840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60394/0 (socket says 172.21.15.33:60394) 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 -- 172.21.15.33:0/1086459016 learned_addr learned my addr 172.21.15.33:0/1086459016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da7205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 0x5632da71ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 -- 172.21.15.33:0/1086459016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 msgr2=0x5632da7205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da7205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 -- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 msgr2=0x5632da71ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 0x5632da71ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fd448640 1 -- 172.21.15.33:0/1086459016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632da6766d0 con 0x5632da7109b0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.566+0000 7fc9fe44a640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 0x5632da71ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9fd448640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80996c0 tx=0x7fc9e80ca090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efab54e273fa38f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8062220 con 0x5632da7109b0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc9e8099e30 con 0x5632da7109b0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e80623c0 con 0x5632da7109b0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e80a1ac0 con 0x5632da7109b0 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 10 client.15420 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5632da638a80 con 0x5632da7109b0 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e80623c0 con 0x5632da7109b0 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc9e8062530 con 0x5632da7109b0 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 10 client.15420 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 20 client.15420 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7806d0 0x5632da782ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 4 client.15420 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.567+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5632da377600 con 0x5632da7806d0 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.568+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7806d0 0x5632da782ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.568+0000 7fc9e17fa640 10 client.15420 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.568+0000 7fc9fdc49640 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7806d0 0x5632da782ac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d858e2138564c28b server_cookie=a7f0f721da204da3 in_seq=0 out_seq=0 2024-01-25T16:08:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.580+0000 7fc9e17fa640 1 -- 172.21.15.33:0/1086459016 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5632da377600 con 0x5632da7806d0 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.580+0000 7fc9e17fa640 10 client.15420 handle_command_reply: tid=0 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.580+0000 7fca02a71740 1 client.15420 shutdown 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 msgr2=0x5632da71ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80996c0 tx=0x7fc9e80ca090 comp rx=0 tx=0).stop 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 shutdown_connections 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5632da7806d0 0x5632da782ac0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5632da719780 0x5632da71ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632da7109b0 0x5632da71ea10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9e17fa640 0 client.15420 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 --2- 172.21.15.33:0/1086459016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5632da710090 0x5632da7205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 >> 172.21.15.33:0/1086459016 conn(0x5632da70a9b0 msgr2=0x5632da70be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 shutdown_connections 2024-01-25T16:08:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 1 -- 172.21.15.33:0/1086459016 wait complete. 2024-01-25T16:08:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fca02a71740 20 client.15420 trim_cache size 0 max 16384 2024-01-25T16:08:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:08:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:08:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:08:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 msgr2=0x7fc9e40a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 0x7fc9e40a3b00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009fd0 tx=0x7fc9f0009dc0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 msgr2=0x7fc9f8073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800d740 tx=0x7fc9e800dc00 comp rx=0 tx=0).stop 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 shutdown_connections 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc9e40a1640 0x7fc9e40a3b00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc9f8073a70 0x7fc9f81a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9f8105fe0 0x7fc9f8073530 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 --2- 172.21.15.33:0/1578868726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9f8100f10 0x7fc9f8072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.581+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 >> 172.21.15.33:0/1578868726 conn(0x7fc9f80fc7b0 msgr2=0x7fc9f80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.582+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 shutdown_connections 2024-01-25T16:08:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:05.582+0000 7fc9ffead640 1 -- 172.21.15.33:0/1578868726 wait complete. 2024-01-25T16:08:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:05 smithi152 ceph-mon[36015]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:07 smithi076 ceph-mon[35642]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:07 smithi033 ceph-mon[31417]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:07.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:07 smithi152 ceph-mon[36015]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 62 op/s 2024-01-25T16:08:09.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:09 smithi076 ceph-mon[35642]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 70 op/s 2024-01-25T16:08:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:09 smithi033 ceph-mon[31417]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 70 op/s 2024-01-25T16:08:09.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:09 smithi152 ceph-mon[36015]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 102 KiB/s wr, 70 op/s 2024-01-25T16:08:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:11 smithi076 ceph-mon[35642]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-01-25T16:08:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:11 smithi033 ceph-mon[31417]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-01-25T16:08:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:11 smithi152 ceph-mon[36015]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-01-25T16:08:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:13 smithi076 ceph-mon[35642]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-01-25T16:08:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:13 smithi033 ceph-mon[31417]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-01-25T16:08:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:13 smithi152 ceph-mon[36015]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 27 op/s 2024-01-25T16:08:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:15 smithi076 ceph-mon[35642]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:15 smithi033 ceph-mon[31417]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:15.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:15 smithi152 ceph-mon[36015]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:17 smithi076 ceph-mon[35642]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:17 smithi033 ceph-mon[31417]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:17.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:17 smithi152 ceph-mon[36015]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 2.7 KiB/s wr, 9 op/s 2024-01-25T16:08:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 -- 172.21.15.33:0/735254378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98380ffe30 msgr2=0x7f9838105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 --2- 172.21.15.33:0/735254378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98380ffe30 0x7f9838105960 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9824009930 tx=0x7f982402bc30 comp rx=0 tx=0).stop 2024-01-25T16:08:18.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 -- 172.21.15.33:0/735254378 shutdown_connections 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 --2- 172.21.15.33:0/735254378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 0x7f983810dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 --2- 172.21.15.33:0/735254378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98380ffe30 0x7f9838105960 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 --2- 172.21.15.33:0/735254378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f98380ff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 -- 172.21.15.33:0/735254378 >> 172.21.15.33:0/735254378 conn(0x7f98380761b0 msgr2=0x7f98380fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 -- 172.21.15.33:0/735254378 shutdown_connections 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.679+0000 7f983eb4c640 1 -- 172.21.15.33:0/735254378 wait complete. 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 Processor -- start 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 -- start start 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f9838197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 0x7f98381977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819c510 con 0x7f983819bb20 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819c680 con 0x7f98380ff480 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983eb4c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983819c7f0 con 0x7f9838106090 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50326/0 (socket says 172.21.15.33:50326) 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983d0e9640 1 -- 172.21.15.33:0/3495986416 learned_addr learned my addr 172.21.15.33:0/3495986416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f983c8e8640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f9838197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.680+0000 7f9837fff640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 0x7f98381977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 -- 172.21.15.33:0/3495986416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 msgr2=0x7f98381977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 0x7f98381977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 -- 172.21.15.33:0/3495986416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 msgr2=0x7f9838197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f9838197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 -- 172.21.15.33:0/3495986416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9824009560 con 0x7f983819bb20 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983c8e8640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f9838197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:18.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983d0e9640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f983000eaa0 tx=0x7f983000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f9835ffb640 1 -- 172.21.15.33:0/3495986416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983000cdb0 con 0x7f983819bb20 2024-01-25T16:08:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f9835ffb640 1 -- 172.21.15.33:0/3495986416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9830005020 con 0x7f983819bb20 2024-01-25T16:08:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98381a3dd0 con 0x7f983819bb20 2024-01-25T16:08:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f9835ffb640 1 -- 172.21.15.33:0/3495986416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300116d0 con 0x7f983819bb20 2024-01-25T16:08:18.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.681+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a42a0 con 0x7f983819bb20 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.683+0000 7f9835ffb640 1 -- 172.21.15.33:0/3495986416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f9830005190 con 0x7f983819bb20 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f9835ffb640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 0x7f98280a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 Processor -- start 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 -- start start 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f983c8e8640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 0x7f98280a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f9835ffb640 1 -- 172.21.15.33:0/3495986416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f98300cd410 con 0x7f983819bb20 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950bc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950c5780 0x556b950c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9502b070 con 0x556b950bc9b0 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b95028760 con 0x556b950bc090 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f98417ec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b95024050 con 0x556b950c5780 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f9837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f9837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50334/0 (socket says 172.21.15.33:50334) 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.684+0000 7f9837fff640 1 -- 172.21.15.33:0/4206600160 learned_addr learned my addr 172.21.15.33:0/4206600160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f983c8e8640 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950bc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f983d0e9640 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950c5780 0x556b950c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f983c8e8640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 0x7f98280a3b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f982c009c30 tx=0x7f982c0117b0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 -- 172.21.15.33:0/4206600160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950c5780 msgr2=0x556b950c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950c5780 0x556b950c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 -- 172.21.15.33:0/4206600160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 msgr2=0x556b950bc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950bc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 -- 172.21.15.33:0/4206600160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b9501be70 con 0x556b950bc9b0 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9837fff640 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9824000c00 tx=0x7f982402c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3794c3c54f894 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9822ffd640 1 -- 172.21.15.33:0/4206600160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98240048c0 con 0x556b950bc9b0 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9822ffd640 1 -- 172.21.15.33:0/4206600160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9824004a20 con 0x556b950bc9b0 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.685+0000 7f9822ffd640 1 -- 172.21.15.33:0/4206600160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005260 con 0x556b950bc9b0 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- 172.21.15.33:0/4206600160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 msgr2=0x556b950c5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9824000c00 tx=0x7f982402c6e0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- 172.21.15.33:0/4206600160 shutdown_connections 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950c5780 0x556b950c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c5240 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 --2- 172.21.15.33:0/4206600160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950bc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- 172.21.15.33:0/4206600160 >> 172.21.15.33:0/4206600160 conn(0x556b950b69b0 msgr2=0x556b950b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- 172.21.15.33:0/4206600160 shutdown_connections 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- 172.21.15.33:0/4206600160 wait complete. 2024-01-25T16:08:18.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 -- start start 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.686+0000 7f98417ec740 1 Processor -- start 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc090 0x556b950c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950c5780 0x556b950c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94fdd1e0 con 0x556b950bc9b0 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94fdce80 con 0x556b950c5780 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b94fa20c0 con 0x556b950bc090 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f98417ec740 10 client.0 _ll_register_callbacks cb 0x556b94fc7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50340/0 (socket says 172.21.15.33:50340) 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 -- 172.21.15.33:0/4034372587 learned_addr learned my addr 172.21.15.33:0/4034372587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f983c8e8640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc090 0x556b950c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f983d0e9640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950c5780 0x556b950c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 -- 172.21.15.33:0/4034372587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc090 msgr2=0x556b950c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc090 0x556b950c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 -- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950c5780 msgr2=0x556b950c3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950c5780 0x556b950c3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 -- 172.21.15.33:0/4034372587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983810e420 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9837fff640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f982402c170 tx=0x7f982402c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431036662e8736df server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.687+0000 7f9817fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005410 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f9824005570 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b9501d040 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982402cc10 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 10 client.15432 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556b950226d0 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b9501d040 con 0x556b950bc9b0 2024-01-25T16:08:18.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f982403c040 con 0x556b950bc9b0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 10 client.15432 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 20 client.15432 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b950d50a0 0x556b950d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 4 client.15432 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.688+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b9501cd60 con 0x556b950d50a0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.689+0000 7f983c8e8640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b950d50a0 0x556b950d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.689+0000 7f983c8e8640 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b950d50a0 0x556b950d7490 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1152bf825a1f43b server_cookie=a6b00dd645fb3a07 in_seq=0 out_seq=0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.689+0000 7f9817fff640 10 client.15432 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f9817fff640 1 -- 172.21.15.33:0/4034372587 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b9501cd60 con 0x556b950d50a0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f9817fff640 10 client.15432 handle_command_reply: tid=0 2024-01-25T16:08:18.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 client.15432 shutdown 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 msgr2=0x556b950c2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f982402c170 tx=0x7f982402c9b0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 shutdown_connections 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b950d50a0 0x556b950d7490 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950c5780 0x556b950c3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950bc9b0 0x556b950c2be0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f9817fff640 0 client.15432 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 --2- 172.21.15.33:0/4034372587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc090 0x556b950c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 >> 172.21.15.33:0/4034372587 conn(0x556b950b69b0 msgr2=0x556b950b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 shutdown_connections 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 1 -- 172.21.15.33:0/4034372587 wait complete. 2024-01-25T16:08:18.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.690+0000 7f98417ec740 20 client.15432 trim_cache size 0 max 16384 2024-01-25T16:08:18.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 Processor -- start 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 -- start start 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b950b20a0 con 0x556b950c5780 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.703+0000 7f98417ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b950c7b70 con 0x556b950bc090 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f98417ec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b950c0c20 con 0x556b950bc9b0 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50348/0 (socket says 172.21.15.33:50348) 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 -- 172.21.15.33:0/296364908 learned_addr learned my addr 172.21.15.33:0/296364908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983c8e8640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f9837fff640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 -- 172.21.15.33:0/296364908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 msgr2=0x556b950c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 -- 172.21.15.33:0/296364908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 msgr2=0x556b950c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 -- 172.21.15.33:0/296364908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b94fbf060 con 0x556b950c5780 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983c8e8640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f983d0e9640 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f983000efd0 tx=0x7f9830096330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0e370e364a1f3d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f98227fc640 1 -- 172.21.15.33:0/296364908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830099f70 con 0x556b950c5780 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f98227fc640 1 -- 172.21.15.33:0/296364908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f983009e0c0 con 0x556b950c5780 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.704+0000 7f98227fc640 1 -- 172.21.15.33:0/296364908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830099460 con 0x556b950c5780 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- 172.21.15.33:0/296364908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 msgr2=0x556b950c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f983000efd0 tx=0x7f9830096330 comp rx=0 tx=0).stop 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- 172.21.15.33:0/296364908 shutdown_connections 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950c1680 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 --2- 172.21.15.33:0/296364908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- 172.21.15.33:0/296364908 >> 172.21.15.33:0/296364908 conn(0x556b950b69b0 msgr2=0x556b950c0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- 172.21.15.33:0/296364908 shutdown_connections 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- 172.21.15.33:0/296364908 wait complete. 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 -- start start 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.705+0000 7f98417ec740 1 Processor -- start 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9512b580 con 0x556b950c5780 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9512b6c0 con 0x556b950bc090 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b9512b800 con 0x556b950bc9b0 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f98417ec740 10 client.0 _ll_register_callbacks cb 0x7f982c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:18.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50364/0 (socket says 172.21.15.33:50364) 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 -- 172.21.15.33:0/2371827540 learned_addr learned my addr 172.21.15.33:0/2371827540 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f9837fff640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983c8e8640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 -- 172.21.15.33:0/2371827540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 msgr2=0x556b950cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 -- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 msgr2=0x556b950caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 -- 172.21.15.33:0/2371827540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98381a42a0 con 0x556b950c5780 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f9837fff640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983c8e8640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f983d0e9640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f98300969e0 tx=0x7f9830000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d991bc507ab0cd server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.706+0000 7f9817fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:18.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9830099f70 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f98300a2300 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9824004a20 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9830099460 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 10 client.15438 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x556b94fe4a80 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9824004a20 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9830018030 con 0x556b950c5780 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 10 client.15438 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:18.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 20 client.15438 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b9512cb70 0x556b9512ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 4 client.15438 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.707+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b94d23600 con 0x556b9512cb70 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.708+0000 7f983c8e8640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b9512cb70 0x556b9512ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.708+0000 7f983c8e8640 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b9512cb70 0x556b9512ef90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e59b4795d487ef5e server_cookie=795370f5c4ee4dce in_seq=0 out_seq=0 2024-01-25T16:08:18.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.708+0000 7f9817fff640 10 client.15438 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f9817fff640 1 -- 172.21.15.33:0/2371827540 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b94d23600 con 0x556b9512cb70 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f9817fff640 10 client.15438 handle_command_reply: tid=0 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 client.15438 shutdown 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 msgr2=0x556b950cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f98300969e0 tx=0x7f9830000f40 comp rx=0 tx=0).stop 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 shutdown_connections 2024-01-25T16:08:18.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b9512cb70 0x556b9512ef90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b950c5780 0x556b950cba00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b950bc9b0 0x556b950cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f9817fff640 0 client.15438 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 --2- 172.21.15.33:0/2371827540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b950bc090 0x556b950caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 >> 172.21.15.33:0/2371827540 conn(0x556b950b69b0 msgr2=0x556b950b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 shutdown_connections 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 1 -- 172.21.15.33:0/2371827540 wait complete. 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.715+0000 7f98417ec740 20 client.15438 trim_cache size 0 max 16384 2024-01-25T16:08:18.720 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:08:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:08:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:08:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:08:18.721 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 msgr2=0x7f98280a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 0x7f98280a3b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f982c009c30 tx=0x7f982c0117b0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 msgr2=0x7f983819bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f983000eaa0 tx=0x7f983000ef60 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 shutdown_connections 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f98280a1690 0x7f98280a3b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f983819bb20 0x7f983819bfd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9838106090 0x7f98381977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 --2- 172.21.15.33:0/3495986416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98380ff480 0x7f9838197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 >> 172.21.15.33:0/3495986416 conn(0x7f98380761b0 msgr2=0x7f98381042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:18.722 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 shutdown_connections 2024-01-25T16:08:18.723 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:18.716+0000 7f983eb4c640 1 -- 172.21.15.33:0/3495986416 wait complete. 2024-01-25T16:08:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:19 smithi076 ceph-mon[35642]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-01-25T16:08:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:19 smithi033 ceph-mon[31417]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-01-25T16:08:19.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:19 smithi152 ceph-mon[36015]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-01-25T16:08:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:21 smithi076 ceph-mon[35642]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-25T16:08:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:21 smithi033 ceph-mon[31417]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-25T16:08:21.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:21 smithi152 ceph-mon[36015]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-01-25T16:08:23.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:23 smithi076 ceph-mon[35642]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-01-25T16:08:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:23 smithi033 ceph-mon[31417]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-01-25T16:08:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:23 smithi152 ceph-mon[36015]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 597 B/s wr, 8 op/s 2024-01-25T16:08:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:25 smithi076 ceph-mon[35642]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:25 smithi033 ceph-mon[31417]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:25 smithi152 ceph-mon[36015]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:27 smithi076 ceph-mon[35642]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:27 smithi033 ceph-mon[31417]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:27 smithi152 ceph-mon[36015]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-01-25T16:08:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:29 smithi076 ceph-mon[35642]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-01-25T16:08:29.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:29 smithi152 ceph-mon[36015]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-01-25T16:08:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:29 smithi033 ceph-mon[31417]: pgmap v22: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-01-25T16:08:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:31 smithi076 ceph-mon[35642]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-01-25T16:08:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:31 smithi033 ceph-mon[31417]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-01-25T16:08:31.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:31 smithi152 ceph-mon[36015]: pgmap v23: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- 172.21.15.33:0/2313267883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f94104b90 msgr2=0x7f4f94104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- 172.21.15.33:0/2313267883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f94104b90 0x7f4f94104f70 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009900 tx=0x7f4f8802bc30 comp rx=0 tx=0).stop 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- 172.21.15.33:0/2313267883 shutdown_connections 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- 172.21.15.33:0/2313267883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94100800 0x7f4f9410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- 172.21.15.33:0/2313267883 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f940ffe40 0x7f4f941002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- 172.21.15.33:0/2313267883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f94104b90 0x7f4f94104f70 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- 172.21.15.33:0/2313267883 >> 172.21.15.33:0/2313267883 conn(0x7f4f94076a40 msgr2=0x7f4f94076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- 172.21.15.33:0/2313267883 shutdown_connections 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- 172.21.15.33:0/2313267883 wait complete. 2024-01-25T16:08:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 Processor -- start 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- start start 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 0x7f4f94197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94104b90 0x7f4f9419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.816+0000 7f4f9add0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419c100 con 0x7f4f940ffe40 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f9add0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419c270 con 0x7f4f94104b90 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f9add0640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9419c430 con 0x7f4f94100800 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44918/0 (socket says 172.21.15.33:44918) 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 -- 172.21.15.33:0/4226145337 learned_addr learned my addr 172.21.15.33:0/4226145337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f9936d640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94104b90 0x7f4f9419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f93fff640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 0x7f4f94197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 -- 172.21.15.33:0/4226145337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 msgr2=0x7f4f94197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 0x7f4f94197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 -- 172.21.15.33:0/4226145337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94104b90 msgr2=0x7f4f9419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94104b90 0x7f4f9419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 -- 172.21.15.33:0/4226145337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f88009560 con 0x7f4f940ffe40 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f93fff640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 0x7f4f94197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005940 tx=0x7f4f88004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f91ffb640 1 -- 172.21.15.33:0/4226145337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88004350 con 0x7f4f940ffe40 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9419c6b0 con 0x7f4f940ffe40 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f91ffb640 1 -- 172.21.15.33:0/4226145337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f880044b0 con 0x7f4f940ffe40 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f91ffb640 1 -- 172.21.15.33:0/4226145337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f88003e10 con 0x7f4f940ffe40 2024-01-25T16:08:31.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.817+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f941a3f90 con 0x7f4f940ffe40 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.820+0000 7f4f91ffb640 1 -- 172.21.15.33:0/4226145337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f4f88005000 con 0x7f4f940ffe40 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.820+0000 7f4f91ffb640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 0x7f4f840a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.820+0000 7f4f93fff640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 0x7f4f840a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.820+0000 7f4f91ffb640 1 -- 172.21.15.33:0/4226145337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f8802de40 con 0x7f4f940ffe40 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 Processor -- start 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f93fff640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 0x7f4f840a3ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f800058c0 tx=0x7f4f80005850 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 -- start start 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e48090 0x559b64e48470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 0x559b64e51240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64db7070 con 0x559b64e51780 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64db4760 con 0x559b64e489b0 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9d991740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64db0050 con 0x559b64e48090 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44930/0 (socket says 172.21.15.33:44930) 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f9936d640 1 -- 172.21.15.33:0/3013943093 learned_addr learned my addr 172.21.15.33:0/3013943093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e48090 0x559b64e48470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.821+0000 7f4f93fff640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 0x559b64e51240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 -- 172.21.15.33:0/3013943093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e48090 msgr2=0x559b64e48470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e48090 0x559b64e48470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 -- 172.21.15.33:0/3013943093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 msgr2=0x559b64e51240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 0x559b64e51240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 -- 172.21.15.33:0/3013943093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b64da7e70 con 0x559b64e51780 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f93fff640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 0x559b64e51240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c010370 tx=0x7f4f8c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa86de9a6ee93b46 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f7effd640 1 -- 172.21.15.33:0/3013943093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c004d20 con 0x559b64e51780 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f7effd640 1 -- 172.21.15.33:0/3013943093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c004e80 con 0x559b64e51780 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.822+0000 7f4f7effd640 1 -- 172.21.15.33:0/3013943093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c005480 con 0x559b64e51780 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- 172.21.15.33:0/3013943093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 msgr2=0x559b64e53b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c010370 tx=0x7f4f8c010830 comp rx=0 tx=0).stop 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- 172.21.15.33:0/3013943093 shutdown_connections 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e53b70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e489b0 0x559b64e51240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3013943093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e48090 0x559b64e48470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- 172.21.15.33:0/3013943093 >> 172.21.15.33:0/3013943093 conn(0x559b64e429b0 msgr2=0x559b64e42d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- 172.21.15.33:0/3013943093 shutdown_connections 2024-01-25T16:08:31.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- 172.21.15.33:0/3013943093 wait complete. 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- start start 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 Processor -- start 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e51780 0x559b64e4f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64d691e0 con 0x559b64e48090 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64d68e80 con 0x559b64e51780 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64d2e0c0 con 0x559b64e489b0 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.823+0000 7f4f9d991740 10 client.0 _ll_register_callbacks cb 0x559b64d53ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44932/0 (socket says 172.21.15.33:44932) 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 -- 172.21.15.33:0/1386690229 learned_addr learned my addr 172.21.15.33:0/1386690229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:31.833 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 -- 172.21.15.33:0/1386690229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 msgr2=0x559b64e4ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f93fff640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f9936d640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e51780 0x559b64e4f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 -- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e51780 msgr2=0x559b64e4f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e51780 0x559b64e4f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 -- 172.21.15.33:0/1386690229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f8c009c40 con 0x559b64e48090 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f93fff640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b8bc0 tx=0x7f4f880bc020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ceefe4f4a100a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f73fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f880bcb20 con 0x559b64e48090 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f880c0810 con 0x559b64e48090 2024-01-25T16:08:31.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b64da8d60 con 0x559b64e48090 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f880c4a50 con 0x559b64e48090 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f9d991740 10 client.15450 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.824+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559b64dae6d0 con 0x559b64e48090 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b64da8d60 con 0x559b64e48090 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f8803c070 con 0x559b64e48090 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 10 client.15450 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 10 client.15450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 20 client.15450 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64e61230 0x559b64e63640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 4 client.15450 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b64aaf600 con 0x559b64e61230 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f93fff640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64e61230 0x559b64e63640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f73fff640 10 client.15450 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.825+0000 7f4f93fff640 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64e61230 0x559b64e63640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fc4bc9d5b75321 server_cookie=f4f1e67996e09902 in_seq=0 out_seq=0 2024-01-25T16:08:31.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f73fff640 1 -- 172.21.15.33:0/1386690229 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b64aaf600 con 0x559b64e61230 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f73fff640 10 client.15450 handle_command_reply: tid=0 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 client.15450 shutdown 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 msgr2=0x559b64e4e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4f880b8bc0 tx=0x7f4f880bc020 comp rx=0 tx=0).stop 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 shutdown_connections 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64e61230 0x559b64e63640 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e51780 0x559b64e4f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f73fff640 0 client.15450 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1386690229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e48090 0x559b64e4e700 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 >> 172.21.15.33:0/1386690229 conn(0x559b64e429b0 msgr2=0x559b64e44700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 shutdown_connections 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.826+0000 7f4f9d991740 1 -- 172.21.15.33:0/1386690229 wait complete. 2024-01-25T16:08:31.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.827+0000 7f4f9d991740 20 client.15450 trim_cache size 0 max 16384 2024-01-25T16:08:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 Processor -- start 2024-01-25T16:08:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 -- start start 2024-01-25T16:08:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e50e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64aaf600 con 0x559b64e51780 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64da8d60 con 0x559b64e48090 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9d991740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64e49040 con 0x559b64e489b0 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44948/0 (socket says 172.21.15.33:44948) 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 -- 172.21.15.33:0/3725062674 learned_addr learned my addr 172.21.15.33:0/3725062674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f93fff640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e50e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 -- 172.21.15.33:0/3725062674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 msgr2=0x559b64e4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 -- 172.21.15.33:0/3725062674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 msgr2=0x559b64e50e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e50e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 -- 172.21.15.33:0/3725062674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b64d4b060 con 0x559b64e51780 2024-01-25T16:08:31.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e50e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.840+0000 7f4f9936d640 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005bd0 tx=0x7f4f8c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d183aa62f33a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f7e7fc640 1 -- 172.21.15.33:0/3725062674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c01f070 con 0x559b64e51780 2024-01-25T16:08:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f7e7fc640 1 -- 172.21.15.33:0/3725062674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f8c0050c0 con 0x559b64e51780 2024-01-25T16:08:31.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f7e7fc640 1 -- 172.21.15.33:0/3725062674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c012770 con 0x559b64e51780 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 -- 172.21.15.33:0/3725062674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 msgr2=0x559b64e4b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c005bd0 tx=0x7f4f8c005c00 comp rx=0 tx=0).stop 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 -- 172.21.15.33:0/3725062674 shutdown_connections 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e4b960 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 --2- 172.21.15.33:0/3725062674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e50e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 -- 172.21.15.33:0/3725062674 >> 172.21.15.33:0/3725062674 conn(0x559b64e429b0 msgr2=0x559b64e4cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 -- 172.21.15.33:0/3725062674 shutdown_connections 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.841+0000 7f4f9d991740 1 -- 172.21.15.33:0/3725062674 wait complete. 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 -- start start 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 Processor -- start 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e58740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e56c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64e57810 con 0x559b64e51780 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64e57980 con 0x559b64e48090 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b64eb7770 con 0x559b64e489b0 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9d991740 10 client.0 _ll_register_callbacks cb 0x7f4f8c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.849 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9936d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44954/0 (socket says 172.21.15.33:44954) 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.842+0000 7f4f9936d640 1 -- 172.21.15.33:0/1333503746 learned_addr learned my addr 172.21.15.33:0/1333503746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e58740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f93fff640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e56c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 -- 172.21.15.33:0/1333503746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 msgr2=0x559b64e56c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e56c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 -- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 msgr2=0x559b64e58740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e58740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 -- 172.21.15.33:0/1333503746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b64da7e70 con 0x559b64e51780 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e58740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9936d640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c010d30 tx=0x7f4f8c010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffacb056462b14e6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f73fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c01f040 con 0x559b64e51780 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f4f8c005640 con 0x559b64e51780 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b64da9040 con 0x559b64e51780 2024-01-25T16:08:31.850 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f8c012510 con 0x559b64e51780 2024-01-25T16:08:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9d991740 10 client.15456 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x559b64d70a80 con 0x559b64e51780 2024-01-25T16:08:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.843+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b64da9040 con 0x559b64e51780 2024-01-25T16:08:31.851 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4f8c02a070 con 0x559b64e51780 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 10 client.15456 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 20 client.15456 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64eb8a00 0x559b64ebadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 4 client.15456 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b64b02c50 con 0x559b64eb8a00 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64eb8a00 0x559b64ebadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f73fff640 10 client.15456 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:31.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.844+0000 7f4f98b6c640 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64eb8a00 0x559b64ebadf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a58ff9cc887491a1 server_cookie=85c79f1f522330ba in_seq=0 out_seq=0 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f73fff640 1 -- 172.21.15.33:0/1333503746 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b64b02c50 con 0x559b64eb8a00 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f73fff640 10 client.15456 handle_command_reply: tid=0 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 client.15456 shutdown 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 msgr2=0x559b64e57140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c010d30 tx=0x7f4f8c010830 comp rx=0 tx=0).stop 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 shutdown_connections 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559b64eb8a00 0x559b64ebadf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559b64e51780 0x559b64e57140 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559b64e489b0 0x559b64e56c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 --2- 172.21.15.33:0/1333503746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559b64e48090 0x559b64e58740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f73fff640 0 client.15456 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 >> 172.21.15.33:0/1333503746 conn(0x559b64e429b0 msgr2=0x559b64e411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 shutdown_connections 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 1 -- 172.21.15.33:0/1333503746 wait complete. 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.852+0000 7f4f9d991740 20 client.15456 trim_cache size 0 max 16384 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:08:31.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 msgr2=0x7f4f840a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 0x7f4f840a3ad0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f800058c0 tx=0x7f4f80005850 comp rx=0 tx=0).stop 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 msgr2=0x7f4f94197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4f88005940 tx=0x7f4f88004e50 comp rx=0 tx=0).stop 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 shutdown_connections 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4f840a1610 0x7f4f840a3ad0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f94104b90 0x7f4f9419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f94100800 0x7f4f94197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 --2- 172.21.15.33:0/4226145337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f940ffe40 0x7f4f94197240 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 >> 172.21.15.33:0/4226145337 conn(0x7f4f94076a40 msgr2=0x7f4f94068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 shutdown_connections 2024-01-25T16:08:31.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:31.853+0000 7f4f9add0640 1 -- 172.21.15.33:0/4226145337 wait complete. 2024-01-25T16:08:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:33 smithi033 ceph-mon[31417]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-25T16:08:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:33 smithi152 ceph-mon[36015]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-25T16:08:33.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:33 smithi076 ceph-mon[35642]: pgmap v24: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-01-25T16:08:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:35 smithi033 ceph-mon[31417]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:35 smithi152 ceph-mon[36015]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:35 smithi076 ceph-mon[35642]: pgmap v25: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:37 smithi033 ceph-mon[31417]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:37 smithi152 ceph-mon[36015]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:37 smithi076 ceph-mon[35642]: pgmap v26: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-01-25T16:08:38.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:38 smithi033 ceph-mon[31417]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-25T16:08:38.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:38 smithi152 ceph-mon[36015]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-25T16:08:38.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:38 smithi076 ceph-mon[35642]: pgmap v27: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-01-25T16:08:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:41 smithi076 ceph-mon[35642]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:41 smithi033 ceph-mon[31417]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:41.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:41 smithi152 ceph-mon[36015]: pgmap v28: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:43.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:43 smithi076 ceph-mon[35642]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-25T16:08:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:43 smithi033 ceph-mon[31417]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-25T16:08:43.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:43 smithi152 ceph-mon[36015]: pgmap v29: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 -- 172.21.15.33:0/1637987785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538100800 msgr2=0x7f853810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1637987785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538100800 0x7f853810ca20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8530009ca0 tx=0x7f853002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 -- 172.21.15.33:0/1637987785 shutdown_connections 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1637987785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538100800 0x7f853810ca20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1637987785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1637987785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538104b90 0x7f8538104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 -- 172.21.15.33:0/1637987785 >> 172.21.15.33:0/1637987785 conn(0x7f8538076a40 msgr2=0x7f8538076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 -- 172.21.15.33:0/1637987785 shutdown_connections 2024-01-25T16:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.989+0000 7f853e4cf640 1 -- 172.21.15.33:0/1637987785 wait complete. 2024-01-25T16:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.990+0000 7f853e4cf640 1 Processor -- start 2024-01-25T16:08:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.990+0000 7f853e4cf640 1 -- start start 2024-01-25T16:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538100800 0x7f8538197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 0x7f853819baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819c0a0 con 0x7f8538104b90 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819c210 con 0x7f85380ffe40 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853e4cf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853819c3d0 con 0x7f8538100800 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f8537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f85377fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538100800 0x7f8538197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f8537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:54466/0 (socket says 172.21.15.33:54466) 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f8537fff640 1 -- 172.21.15.33:0/1079801768 learned_addr learned my addr 172.21.15.33:0/1079801768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.991+0000 7f853ca6c640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 0x7f853819baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 -- 172.21.15.33:0/1079801768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538100800 msgr2=0x7f8538197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538100800 0x7f8538197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 -- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 msgr2=0x7f853819baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 0x7f853819baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 -- 172.21.15.33:0/1079801768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c009490 con 0x7f85380ffe40 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f853ca6c640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 0x7f853819baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f8537fff640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f852800e9f0 tx=0x7f852800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f85357fa640 1 -- 172.21.15.33:0/1079801768 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528009720 con 0x7f85380ffe40 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f85357fa640 1 -- 172.21.15.33:0/1079801768 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852800cd80 con 0x7f85380ffe40 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.992+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8530009900 con 0x7f85380ffe40 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.993+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85381a3d50 con 0x7f85380ffe40 2024-01-25T16:08:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.993+0000 7f85357fa640 1 -- 172.21.15.33:0/1079801768 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528010430 con 0x7f85380ffe40 2024-01-25T16:08:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f85357fa640 1 -- 172.21.15.33:0/1079801768 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7f8528005020 con 0x7f85380ffe40 2024-01-25T16:08:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 Processor -- start 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 -- start start 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091f4a0 0x55f90091f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90088a070 con 0x55f900928b90 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f900887760 con 0x55f90091fdc0 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.996+0000 7f854116c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f900883050 con 0x55f90091f4a0 2024-01-25T16:08:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f853ca6c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091f4a0 0x55f90091f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:54468/0 (socket says 172.21.15.33:54468) 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 -- 172.21.15.33:0/2844561314 learned_addr learned my addr 172.21.15.33:0/2844561314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f85377fe640 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 -- 172.21.15.33:0/2844561314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091f4a0 msgr2=0x55f90091f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091f4a0 0x55f90091f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 -- 172.21.15.33:0/2844561314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 msgr2=0x55f90092af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 -- 172.21.15.33:0/2844561314 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f90087ae70 con 0x55f90091fdc0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f8537fff640 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85280cab80 tx=0x7f85280d5430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1c4722ee93ad8b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f851e7fc640 1 -- 172.21.15.33:0/2844561314 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280cf310 con 0x55f90091fdc0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f851e7fc640 1 -- 172.21.15.33:0/2844561314 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85280cf470 con 0x55f90091fdc0 2024-01-25T16:08:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.997+0000 7f851e7fc640 1 -- 172.21.15.33:0/2844561314 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280dc7c0 con 0x55f90091fdc0 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f85357fa640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 0x7f85200a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f853ca6c640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 0x7f85200a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f85357fa640 1 -- 172.21.15.33:0/1079801768 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f8528014070 con 0x7f85380ffe40 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f854116c740 1 -- 172.21.15.33:0/2844561314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 msgr2=0x55f900928650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f854116c740 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f85280cab80 tx=0x7f85280d5430 comp rx=0 tx=0).stop 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.998+0000 7f853ca6c640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 0x7f85200a3b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8530004af0 tx=0x7f8530005c50 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 -- 172.21.15.33:0/2844561314 shutdown_connections 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900928650 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 --2- 172.21.15.33:0/2844561314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091f4a0 0x55f90091f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 -- 172.21.15.33:0/2844561314 >> 172.21.15.33:0/2844561314 conn(0x55f900919dc0 msgr2=0x55f90091a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 -- 172.21.15.33:0/2844561314 shutdown_connections 2024-01-25T16:08:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 -- 172.21.15.33:0/2844561314 wait complete. 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 -- start start 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:44.999+0000 7f854116c740 1 Processor -- start 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41158/0 (socket says 172.21.15.33:41158) 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 -- 172.21.15.33:0/3987530409 learned_addr learned my addr 172.21.15.33:0/3987530409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900926020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 0x55f900926560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90083c1e0 con 0x55f90091f4a0 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90083be80 con 0x55f90091fdc0 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9008010c0 con 0x55f900928b90 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f85377fe640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900926020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 -- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 msgr2=0x55f900926560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f853ca6c640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 0x55f900926560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 0x55f900926560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 -- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 msgr2=0x55f900926020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.000+0000 7f8537fff640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900926020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f8537fff640 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85381a3d50 con 0x55f90091f4a0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f853ca6c640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 0x55f900926560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f8537fff640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8528096550 tx=0x7f85280cd600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfb99c9606d5299 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f85177fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f85177fe640 1 -- 172.21.15.33:0/3987530409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528096390 con 0x55f90091f4a0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f854116c740 10 client.0 _ll_register_callbacks cb 0x55f900826ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f85177fe640 1 -- 172.21.15.33:0/3987530409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f852809a750 con 0x55f90091f4a0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f90087c040 con 0x55f90091f4a0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f85177fe640 1 -- 172.21.15.33:0/3987530409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8528099e20 con 0x55f90091f4a0 2024-01-25T16:08:45.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f854116c740 10 client.15468 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.001+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=22}) v3 -- 0x55f9008816d0 con 0x55f90091f4a0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f85177fe640 1 -- 172.21.15.33:0/3987530409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f852801f060 con 0x55f90091f4a0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 10 client.15468 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 10 client.15468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 20 client.15468 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f9009383f0 0x55f90093a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 4 client.15468 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f90087c040 con 0x55f9009383f0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f85377fe640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f9009383f0 0x55f90093a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f85177fe640 10 client.15468 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.002+0000 7f85377fe640 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f9009383f0 0x55f90093a890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e22c135feece95c server_cookie=dbe0419ed06ac6fd in_seq=0 out_seq=0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.003+0000 7f85177fe640 1 -- 172.21.15.33:0/3987530409 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f90087c040 con 0x55f9009383f0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.003+0000 7f85177fe640 10 client.15468 handle_command_reply: tid=0 2024-01-25T16:08:45.011 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.003+0000 7f854116c740 1 client.15468 shutdown 2024-01-25T16:08:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.003+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 msgr2=0x55f900925ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.012 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.003+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8528096550 tx=0x7f85280cd600 comp rx=0 tx=0).stop 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 shutdown_connections 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f9009383f0 0x55f90093a890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f900928b90 0x55f900926560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f85177fe640 0 client.15468 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f90091fdc0 0x55f900926020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 --2- 172.21.15.33:0/3987530409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f90091f4a0 0x55f900925ae0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 >> 172.21.15.33:0/3987530409 conn(0x55f900919dc0 msgr2=0x55f90091bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 shutdown_connections 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 1 -- 172.21.15.33:0/3987530409 wait complete. 2024-01-25T16:08:45.013 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.004+0000 7f854116c740 20 client.15468 trim_cache size 0 max 16384 2024-01-25T16:08:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.030+0000 7f854116c740 1 Processor -- start 2024-01-25T16:08:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.030+0000 7f854116c740 1 -- start start 2024-01-25T16:08:45.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.030+0000 7f854116c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f900928210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.030+0000 7f854116c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f854116c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f900923280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f85377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f85377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41170/0 (socket says 172.21.15.33:41170) 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f85377fe640 1 -- 172.21.15.33:0/784373701 learned_addr learned my addr 172.21.15.33:0/784373701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90093faa0 con 0x55f900928b90 2024-01-25T16:08:45.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90092b300 con 0x55f9009383f0 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f900919bf0 con 0x55f90091fdc0 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f900928210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f853ca6c640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f900923280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 -- 172.21.15.33:0/784373701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 msgr2=0x55f900923280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f900923280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 -- 172.21.15.33:0/784373701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 msgr2=0x55f900922d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f8537fff640 1 -- 172.21.15.33:0/784373701 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f90081e060 con 0x55f90091fdc0 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.031+0000 7f85377fe640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:45.038 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.032+0000 7f8537fff640 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f900928210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85280cab80 tx=0x7f8528019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39fe4d7be567a79 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.032+0000 7f851dffb640 1 -- 172.21.15.33:0/784373701 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85280d0ae0 con 0x55f90091fdc0 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.032+0000 7f851dffb640 1 -- 172.21.15.33:0/784373701 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85280d4070 con 0x55f90091fdc0 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.032+0000 7f851dffb640 1 -- 172.21.15.33:0/784373701 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528099e20 con 0x55f90091fdc0 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 msgr2=0x55f900928210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f900928210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f85280cab80 tx=0x7f8528019000 comp rx=0 tx=0).stop 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 shutdown_connections 2024-01-25T16:08:45.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f900923280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f900922d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 --2- 172.21.15.33:0/784373701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f900928210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 >> 172.21.15.33:0/784373701 conn(0x55f900919dc0 msgr2=0x55f90091a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 shutdown_connections 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- 172.21.15.33:0/784373701 wait complete. 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.033+0000 7f854116c740 1 -- start start 2024-01-25T16:08:45.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 Processor -- start 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f90092e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f90092ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90098e980 con 0x55f900928b90 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90098eac0 con 0x55f9009383f0 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f90098ec00 con 0x55f90091fdc0 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f854116c740 10 client.0 _ll_register_callbacks cb 0x7f85280d0c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f85377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f85377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41186/0 (socket says 172.21.15.33:41186) 2024-01-25T16:08:45.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f85377fe640 1 -- 172.21.15.33:0/665312639 learned_addr learned my addr 172.21.15.33:0/665312639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.034+0000 7f8537fff640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f90092e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f853ca6c640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f90092ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 -- 172.21.15.33:0/665312639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 msgr2=0x55f90092e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f90092e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 -- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 msgr2=0x55f90092ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f90092ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 -- 172.21.15.33:0/665312639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f852c009bb0 con 0x55f900928b90 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85377fe640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55f90091af50 tx=0x7f852c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0731693c5d5b980 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85177fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.035+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c017040 con 0x55f900928b90 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f852c005220 con 0x55f900928b90 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85280cf470 con 0x55f900928b90 2024-01-25T16:08:45.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f852c0053e0 con 0x55f900928b90 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 10 client.15474 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55f9008816d0 con 0x55f900928b90 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f852c033080 con 0x55f900928b90 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f852c02f6c0 con 0x55f900928b90 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 10 client.15474 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 10 client.15474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 20 client.15474 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f900990490 0x55f9009928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 4 client.15474 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.036+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f90087bd60 con 0x55f900990490 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.037+0000 7f8537fff640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f900990490 0x55f9009928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.037+0000 7f85177fe640 10 client.15474 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.037+0000 7f8537fff640 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f900990490 0x55f9009928b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca5871fe3fe94d77 server_cookie=1f9188c719100234 in_seq=0 out_seq=0 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f85177fe640 1 -- 172.21.15.33:0/665312639 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f90087bd60 con 0x55f900990490 2024-01-25T16:08:45.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f85177fe640 10 client.15474 handle_command_reply: tid=0 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 client.15474 shutdown 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 msgr2=0x55f90092e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x55f90091af50 tx=0x7f852c010800 comp rx=0 tx=0).stop 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 shutdown_connections 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f900990490 0x55f9009928b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f9009383f0 0x55f90092ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f900928b90 0x55f90092e9d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 --2- 172.21.15.33:0/665312639 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f90091fdc0 0x55f90092e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f85177fe640 0 client.15474 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.041+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 >> 172.21.15.33:0/665312639 conn(0x55f900919dc0 msgr2=0x55f9009243b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.042+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 shutdown_connections 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.042+0000 7f854116c740 1 -- 172.21.15.33:0/665312639 wait complete. 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.042+0000 7f854116c740 20 client.15474 trim_cache size 0 max 16384 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:08:45.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 msgr2=0x7f85200a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 0x7f85200a3b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8530004af0 tx=0x7f8530005c50 comp rx=0 tx=0).stop 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 msgr2=0x7f85381971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f852800e9f0 tx=0x7f852800edf0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 shutdown_connections 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85200a1690 0x7f85200a3b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8538104b90 0x7f853819baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8538100800 0x7f8538197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 --2- 172.21.15.33:0/1079801768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85380ffe40 0x7f85381971e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 >> 172.21.15.33:0/1079801768 conn(0x7f8538076a40 msgr2=0x7f85380685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 shutdown_connections 2024-01-25T16:08:45.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:45.043+0000 7f853e4cf640 1 -- 172.21.15.33:0/1079801768 wait complete. 2024-01-25T16:08:45.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:45 smithi076 ceph-mon[35642]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:45 smithi033 ceph-mon[31417]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:45 smithi152 ceph-mon[36015]: pgmap v30: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:47 smithi076 ceph-mon[35642]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:47 smithi033 ceph-mon[31417]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:47 smithi152 ceph-mon[36015]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-01-25T16:08:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:49 smithi076 ceph-mon[35642]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-25T16:08:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:49 smithi033 ceph-mon[31417]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-25T16:08:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:49 smithi152 ceph-mon[36015]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-01-25T16:08:51.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:51 smithi076 ceph-mon[35642]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:51 smithi033 ceph-mon[31417]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:51.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:51 smithi152 ceph-mon[36015]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:53.314 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:53 smithi033 ceph-mon[31417]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:08:53.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:53 smithi076 ceph-mon[35642]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:08:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:53 smithi152 ceph-mon[36015]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:08:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:55 smithi076 ceph-mon[35642]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:55 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:08:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:55 smithi033 ceph-mon[31417]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:55 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:08:55.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:55 smithi152 ceph-mon[36015]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:55.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:55 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:08:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:08:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:08:56.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:56 smithi033 ceph-mon[31417]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:56 smithi152 ceph-mon[36015]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:08:56.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:08:56.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:08:56.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:56 smithi076 ceph-mon[35642]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-01-25T16:08:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 -- 172.21.15.33:0/965100325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef480739c0 msgr2=0x7fef4810d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 --2- 172.21.15.33:0/965100325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef480739c0 0x7fef4810d040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fef30009900 tx=0x7fef3002bcd0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 -- 172.21.15.33:0/965100325 shutdown_connections 2024-01-25T16:08:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 --2- 172.21.15.33:0/965100325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef4810d580 0x7fef4810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 --2- 172.21.15.33:0/965100325 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef480739c0 0x7fef4810d040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 --2- 172.21.15.33:0/965100325 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480730a0 0x7fef48073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 -- 172.21.15.33:0/965100325 >> 172.21.15.33:0/965100325 conn(0x7fef480fc7d0 msgr2=0x7fef480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 -- 172.21.15.33:0/965100325 shutdown_connections 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.135+0000 7fef4cc44640 1 -- 172.21.15.33:0/965100325 wait complete. 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 Processor -- start 2024-01-25T16:08:58.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 -- start start 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef480730a0 0x7fef48199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 0x7fef48199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819e2d0 con 0x7fef4810d580 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819e440 con 0x7fef480730a0 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef4cc44640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef4819e600 con 0x7fef480739c0 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef46d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef46d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51746/0 (socket says 172.21.15.33:51746) 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef46d9d640 1 -- 172.21.15.33:0/2719897474 learned_addr learned my addr 172.21.15.33:0/2719897474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.136+0000 7fef45d9b640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 0x7fef48199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef4659c640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef480730a0 0x7fef48199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 -- 172.21.15.33:0/2719897474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 msgr2=0x7fef48199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 0x7fef48199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 -- 172.21.15.33:0/2719897474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef480730a0 msgr2=0x7fef48199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef480730a0 0x7fef48199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 -- 172.21.15.33:0/2719897474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef30009560 con 0x7fef4810d580 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef45d9b640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 0x7fef48199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef46d9d640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fef4000eaa0 tx=0x7fef4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef3f7fe640 1 -- 172.21.15.33:0/2719897474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4000cdb0 con 0x7fef4810d580 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef3f7fe640 1 -- 172.21.15.33:0/2719897474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef40005020 con 0x7fef4810d580 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4819e8e0 con 0x7fef4810d580 2024-01-25T16:08:58.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef3f7fe640 1 -- 172.21.15.33:0/2719897474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef400116d0 con 0x7fef4810d580 2024-01-25T16:08:58.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.137+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef481a6160 con 0x7fef4810d580 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.140+0000 7fef3f7fe640 1 -- 172.21.15.33:0/2719897474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7fef40005190 con 0x7fef4810d580 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 Processor -- start 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 -- start start 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7458470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a7463b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a73c7070 con 0x5599a7458090 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a73c4760 con 0x5599a74589b0 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4f805740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a73c0050 con 0x5599a7461780 2024-01-25T16:08:58.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef3f7fe640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 0x7fef340a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef46d9d640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 0x7fef340a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.141+0000 7fef4659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:52304/0 (socket says 172.21.15.33:52304) 2024-01-25T16:08:58.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 -- 172.21.15.33:0/3433415088 learned_addr learned my addr 172.21.15.33:0/3433415088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:58.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7458470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef3f7fe640 1 -- 172.21.15.33:0/2719897474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef400cd2d0 con 0x7fef4810d580 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 -- 172.21.15.33:0/3433415088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 msgr2=0x5599a7463b70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a7463b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 -- 172.21.15.33:0/3433415088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 msgr2=0x5599a7458470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7458470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 -- 172.21.15.33:0/3433415088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a73b7e70 con 0x5599a74589b0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7458470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef46d9d640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 0x7fef340a3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef400ca910 tx=0x7fef40009270 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.142+0000 7fef4659c640 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d20 tx=0x7fef38011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acad2dd3888537f3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.143+0000 7fef3cff9640 1 -- 172.21.15.33:0/3433415088 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38019630 con 0x5599a74589b0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.143+0000 7fef3cff9640 1 -- 172.21.15.33:0/3433415088 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef38019c50 con 0x5599a74589b0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.143+0000 7fef3cff9640 1 -- 172.21.15.33:0/3433415088 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800ec30 con 0x5599a74589b0 2024-01-25T16:08:58.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 -- 172.21.15.33:0/3433415088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 msgr2=0x5599a7461240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef38009d20 tx=0x7fef38011810 comp rx=0 tx=0).stop 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 -- 172.21.15.33:0/3433415088 shutdown_connections 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a7463b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a7461240 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 --2- 172.21.15.33:0/3433415088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7458470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 -- 172.21.15.33:0/3433415088 >> 172.21.15.33:0/3433415088 conn(0x5599a74529b0 msgr2=0x5599a7452d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 -- 172.21.15.33:0/3433415088 shutdown_connections 2024-01-25T16:08:58.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.144+0000 7fef4f805740 1 -- 172.21.15.33:0/3433415088 wait complete. 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 -- start start 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 Processor -- start 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a745e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a7461780 0x5599a745f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a73791e0 con 0x5599a74589b0 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a7378e80 con 0x5599a7461780 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a733e0c0 con 0x5599a7458090 2024-01-25T16:08:58.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51764/0 (socket says 172.21.15.33:51764) 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4659c640 1 -- 172.21.15.33:0/3703804070 learned_addr learned my addr 172.21.15.33:0/3703804070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a7461780 0x5599a745f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.145+0000 7fef4f805740 10 client.0 _ll_register_callbacks cb 0x5599a7363ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 -- 172.21.15.33:0/3703804070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 msgr2=0x5599a745e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a745e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 -- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a7461780 msgr2=0x5599a745f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a7461780 0x5599a745f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 -- 172.21.15.33:0/3703804070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef38009950 con 0x5599a74589b0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef4659c640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e910 tx=0x7fef38005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3174a14afcd654 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.146+0000 7fef2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.147+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38004240 con 0x5599a74589b0 2024-01-25T16:08:58.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.147+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef380043a0 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.147+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599a73b9040 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef3800e9b0 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 10 client.15486 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5599a73be6d0 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599a73b9040 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef380035f0 con 0x5599a74589b0 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 10 client.15486 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 20 client.15486 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a7471500 0x5599a7473920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 4 client.15486 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.148+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599a73b8d60 con 0x5599a7471500 2024-01-25T16:08:58.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.149+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a7471500 0x5599a7473920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.149+0000 7fef2dffb640 10 client.15486 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.149+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a7471500 0x5599a7473920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4eee96c8eaa780 server_cookie=2915c4cf8401bfec in_seq=0 out_seq=0 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef2dffb640 1 -- 172.21.15.33:0/3703804070 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599a73b8d60 con 0x5599a7471500 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef2dffb640 10 client.15486 handle_command_reply: tid=0 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 client.15486 shutdown 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 msgr2=0x5599a745ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef3800e910 tx=0x7fef38005cc0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 shutdown_connections 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a7471500 0x5599a7473920 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a7461780 0x5599a745f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a74589b0 0x5599a745ed20 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 --2- 172.21.15.33:0/3703804070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a745e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 >> 172.21.15.33:0/3703804070 conn(0x5599a74529b0 msgr2=0x5599a7454700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef2dffb640 0 client.15486 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 shutdown_connections 2024-01-25T16:08:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 1 -- 172.21.15.33:0/3703804070 wait complete. 2024-01-25T16:08:58.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.150+0000 7fef4f805740 20 client.15486 trim_cache size 0 max 16384 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 Processor -- start 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 -- start start 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a745f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a745d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a7457370 con 0x5599a7458090 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a745a3f0 con 0x5599a74589b0 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4f805740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a74712b0 con 0x5599a7461780 2024-01-25T16:08:58.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef45d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef45d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51774/0 (socket says 172.21.15.33:51774) 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef45d9b640 1 -- 172.21.15.33:0/3371421014 learned_addr learned my addr 172.21.15.33:0/3371421014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.174+0000 7fef4659c640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a745f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a745d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 -- 172.21.15.33:0/3371421014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 msgr2=0x5599a745d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a745d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 -- 172.21.15.33:0/3371421014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 msgr2=0x5599a7460f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 -- 172.21.15.33:0/3371421014 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599a735b060 con 0x5599a74589b0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef45d9b640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef4659c640 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a745f080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef38003a00 tx=0x7fef38009ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9519f58f622e5f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef2ffff640 1 -- 172.21.15.33:0/3371421014 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38009bf0 con 0x5599a74589b0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef2ffff640 1 -- 172.21.15.33:0/3371421014 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef38011d10 con 0x5599a74589b0 2024-01-25T16:08:58.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.175+0000 7fef2ffff640 1 -- 172.21.15.33:0/3371421014 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38005510 con 0x5599a74589b0 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 -- 172.21.15.33:0/3371421014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 msgr2=0x5599a745f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a745f080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fef38003a00 tx=0x7fef38009ed0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 -- 172.21.15.33:0/3371421014 shutdown_connections 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7461780 0x5599a745d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a745f080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 --2- 172.21.15.33:0/3371421014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7458090 0x5599a7460f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 -- 172.21.15.33:0/3371421014 >> 172.21.15.33:0/3371421014 conn(0x5599a74529b0 msgr2=0x5599a74782a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 -- 172.21.15.33:0/3371421014 shutdown_connections 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.176+0000 7fef4f805740 1 -- 172.21.15.33:0/3371421014 wait complete. 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 -- start start 2024-01-25T16:08:58.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 Processor -- start 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a74664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a74669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a7467090 con 0x5599a7461780 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a74c6f60 con 0x5599a74589b0 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599a74c70a0 con 0x5599a7458090 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4f805740 10 client.0 _ll_register_callbacks cb 0x7fef38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef45d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef45d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51786/0 (socket says 172.21.15.33:51786) 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef45d9b640 1 -- 172.21.15.33:0/4257434460 learned_addr learned my addr 172.21.15.33:0/4257434460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef4659c640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a74669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.177+0000 7fef45d9b640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a74664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 -- 172.21.15.33:0/4257434460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 msgr2=0x5599a74664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a74664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 -- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 msgr2=0x5599a74669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a74669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 -- 172.21.15.33:0/4257434460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef48110620 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef4659c640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a74669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef45d9b640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef30009230 tx=0x7fef3002c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09e696c1dbf7f93 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3002ddc0 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fef30000e40 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef38019c50 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef30004ef0 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef4f805740 10 client.15492 fetch_fsmap learned FSMap version 22 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x5599a73be6d0 con 0x5599a7461780 2024-01-25T16:08:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.178+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef38019c50 con 0x5599a7461780 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 22) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef3003c040 con 0x5599a7461780 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 10 client.15492 fetch_fsmap finished waiting for FSMap version 22 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 20 client.15492 populate_metadata read hostname 'smithi033' 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a74c8890 0x5599a74cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 4 client.15492 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599a70bf600 con 0x5599a74c8890 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4659c640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a74c8890 0x5599a74cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef2dffb640 10 client.15492 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:58.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.179+0000 7fef4659c640 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a74c8890 0x5599a74cacb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d454ba42bbed3cb server_cookie=f3c6b2ec25024868 in_seq=0 out_seq=0 2024-01-25T16:08:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.183+0000 7fef2dffb640 1 -- 172.21.15.33:0/4257434460 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599a70bf600 con 0x5599a74c8890 2024-01-25T16:08:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.183+0000 7fef2dffb640 10 client.15492 handle_command_reply: tid=0 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.183+0000 7fef4f805740 1 client.15492 shutdown 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 msgr2=0x5599a74c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef30009230 tx=0x7fef3002c3d0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 shutdown_connections 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599a74c8890 0x5599a74cacb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599a7461780 0x5599a74c6a20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599a74589b0 0x5599a74669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 --2- 172.21.15.33:0/4257434460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599a7458090 0x5599a74664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 >> 172.21.15.33:0/4257434460 conn(0x5599a74529b0 msgr2=0x5599a745d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef2dffb640 0 client.15492 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 shutdown_connections 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 1 -- 172.21.15.33:0/4257434460 wait complete. 2024-01-25T16:08:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.184+0000 7fef4f805740 20 client.15492 trim_cache size 0 max 16384 2024-01-25T16:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:08:58.189 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 msgr2=0x7fef340a3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 0x7fef340a3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fef400ca910 tx=0x7fef40009270 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 msgr2=0x7fef4819dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fef4000eaa0 tx=0x7fef4000ef60 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 shutdown_connections 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fef340a1690 0x7fef340a3b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef4810d580 0x7fef4819dcd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fef480739c0 0x7fef48199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 --2- 172.21.15.33:0/2719897474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef480730a0 0x7fef48199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:08:58.190 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 >> 172.21.15.33:0/2719897474 conn(0x7fef480fc7d0 msgr2=0x7fef480fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:08:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 shutdown_connections 2024-01-25T16:08:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:08:58.185+0000 7fef4cc44640 1 -- 172.21.15.33:0/2719897474 wait complete. 2024-01-25T16:08:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:08:59 smithi076 ceph-mon[35642]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:08:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:08:59 smithi033 ceph-mon[31417]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:08:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:08:59 smithi152 ceph-mon[36015]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-01-25T16:09:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:01 smithi076 ceph-mon[35642]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:09:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:01 smithi033 ceph-mon[31417]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:09:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:01 smithi152 ceph-mon[36015]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-01-25T16:09:02.237 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 98% (12021/12180) Updating files: 99% (12059/12180) Updating files: 100% (12180/12180) Updating files: 100% (12180/12180), done. 2024-01-25T16:09:02.578 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '661946e7044723130c03627323f6a879a9848cb5'. 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-25T16:09:02.579 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-01-25T16:09:02.580 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 661946e7044 Merge PR #53901 into wip-vshankar-testing-20240125.044502 2024-01-25T16:09:02.582 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-25T16:09:02.648 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-25T16:09:02.650 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-25T16:09:02.650 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-25T16:09:03.364 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-25T16:09:03.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:03 smithi076 ceph-mon[35642]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-01-25T16:09:03.440 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-25T16:09:03.479 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-25T16:09:03.481 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-25T16:09:03.481 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-25T16:09:03.519 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-25T16:09:03.522 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T16:09:03.522 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-25T16:09:03.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:03 smithi033 ceph-mon[31417]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-01-25T16:09:03.577 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-01-25T16:09:03.579 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-01-25T16:09:03.579 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-01-25T16:09:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:03 smithi152 ceph-mon[36015]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-01-25T16:09:03.638 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 1 2024-01-25T16:09:03.648 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-25T16:09:03.648 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-01-25T16:09:03.648 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-25T16:09:03.652 INFO:tasks.workunit.client.0.smithi033.stdout:0 of 1 processes prepared for launch 0 sec 2024-01-25T16:09:03.752 INFO:tasks.workunit.client.0.smithi033.stdout:1 of 1 processes prepared for launch 0 sec 2024-01-25T16:09:03.752 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-01-25T16:09:04.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 3343 35.38 MB/sec warmup 1 sec latency 6.259 ms 2024-01-25T16:09:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:05 smithi076 ceph-mon[35642]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:05 smithi033 ceph-mon[31417]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:05.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:05 smithi152 ceph-mon[36015]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:05.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 6116 24.01 MB/sec warmup 2 sec latency 7.831 ms 2024-01-25T16:09:06.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 8694 21.46 MB/sec warmup 3 sec latency 6.938 ms 2024-01-25T16:09:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:07 smithi076 ceph-mon[35642]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:07 smithi033 ceph-mon[31417]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:07 smithi152 ceph-mon[36015]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-01-25T16:09:07.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 11331 19.63 MB/sec warmup 4 sec latency 24.241 ms 2024-01-25T16:09:08.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:08 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:08.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:08 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:08.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:08 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:08.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 12988 17.04 MB/sec warmup 5 sec latency 587.541 ms 2024-01-25T16:09:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:09 smithi033 ceph-mon[31417]: pgmap v42: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-01-25T16:09:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:09 smithi152 ceph-mon[36015]: pgmap v42: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-01-25T16:09:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:09 smithi076 ceph-mon[35642]: pgmap v42: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 354 KiB/s wr, 63 op/s 2024-01-25T16:09:09.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 12988 14.20 MB/sec warmup 6 sec latency 1587.638 ms 2024-01-25T16:09:10.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13042 12.19 MB/sec warmup 7 sec latency 2299.698 ms 2024-01-25T16:09:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 -- 172.21.15.33:0/2665231438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 msgr2=0x7ff0381022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 --2- 172.21.15.33:0/2665231438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff0381022b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff028009900 tx=0x7ff02802bc30 comp rx=0 tx=0).stop 2024-01-25T16:09:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 -- 172.21.15.33:0/2665231438 shutdown_connections 2024-01-25T16:09:11.291 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 --2- 172.21.15.33:0/2665231438 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff03810b340 0x7ff03810d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 --2- 172.21.15.33:0/2665231438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0381027f0 0x7ff03810acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 --2- 172.21.15.33:0/2665231438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff0381022b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 -- 172.21.15.33:0/2665231438 >> 172.21.15.33:0/2665231438 conn(0x7ff038076d20 msgr2=0x7ff038077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 -- 172.21.15.33:0/2665231438 shutdown_connections 2024-01-25T16:09:11.292 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.285+0000 7ff03dff5640 1 -- 172.21.15.33:0/2665231438 wait complete. 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 Processor -- start 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 -- start start 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0381027f0 0x7ff0381976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 0x7ff03819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819c050 con 0x7ff038101ed0 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819c1c0 con 0x7ff03810b340 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.286+0000 7ff03dff5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03819c380 con 0x7ff0381027f0 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34642/0 (socket says 172.21.15.33:34642) 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 -- 172.21.15.33:0/884549460 learned_addr learned my addr 172.21.15.33:0/884549460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:11.293 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff036ffd640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0381027f0 0x7ff0381976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff037fff640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 0x7ff03819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 -- 172.21.15.33:0/884549460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0381027f0 msgr2=0x7ff0381976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0381027f0 0x7ff0381976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 -- 172.21.15.33:0/884549460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 msgr2=0x7ff03819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 0x7ff03819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 -- 172.21.15.33:0/884549460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028009560 con 0x7ff038101ed0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff037fff640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 0x7ff03819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff0377fe640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff028006010 tx=0x7ff028004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff034ff9640 1 -- 172.21.15.33:0/884549460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0280042d0 con 0x7ff038101ed0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff034ff9640 1 -- 172.21.15.33:0/884549460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff028004430 con 0x7ff038101ed0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03819c600 con 0x7ff038101ed0 2024-01-25T16:09:11.294 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff034ff9640 1 -- 172.21.15.33:0/884549460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028003ca0 con 0x7ff038101ed0 2024-01-25T16:09:11.295 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.287+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a3ee0 con 0x7ff038101ed0 2024-01-25T16:09:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff034ff9640 1 -- 172.21.15.33:0/884549460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 136128+0+0 (secure 0 0 0) 0x7ff028005000 con 0x7ff038101ed0 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 Processor -- start 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 -- start start 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff034ff9640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 0x7ff0200a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fdf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9f4e070 con 0x5608d9fe8780 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9f4b760 con 0x5608d9fdf090 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff040bbc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9f47050 con 0x5608d9fdf9b0 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff037fff640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 0x7ff0200a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff034ff9640 1 -- 172.21.15.33:0/884549460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff0280ef9c0 con 0x7ff038101ed0 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff037fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff037fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34650/0 (socket says 172.21.15.33:34650) 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff037fff640 1 -- 172.21.15.33:0/1916592559 learned_addr learned my addr 172.21.15.33:0/1916592559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.290+0000 7ff036ffd640 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff0377fe640 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fdf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 -- 172.21.15.33:0/1916592559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fdf9b0 msgr2=0x5608d9fe8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 -- 172.21.15.33:0/1916592559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 msgr2=0x5608d9fdf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fdf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 -- 172.21.15.33:0/1916592559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d9f3ee70 con 0x5608d9fe8780 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 0x7ff0200a3ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff030014d40 tx=0x7ff030009760 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff037fff640 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff030011c40 tx=0x7ff03001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7f256bd6e8de31 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff01dffb640 1 -- 172.21.15.33:0/1916592559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030014070 con 0x5608d9fe8780 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff01dffb640 1 -- 172.21.15.33:0/1916592559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03001a040 con 0x5608d9fe8780 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.291+0000 7ff01dffb640 1 -- 172.21.15.33:0/1916592559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030003bd0 con 0x5608d9fe8780 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- 172.21.15.33:0/1916592559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 msgr2=0x5608d9feab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff030011c40 tx=0x7ff03001b000 comp rx=0 tx=0).stop 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- 172.21.15.33:0/1916592559 shutdown_connections 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fe8780 0x5608d9feab70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 --2- 172.21.15.33:0/1916592559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fdf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- 172.21.15.33:0/1916592559 >> 172.21.15.33:0/1916592559 conn(0x5608d9fd99b0 msgr2=0x5608d9fd9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- 172.21.15.33:0/1916592559 shutdown_connections 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- 172.21.15.33:0/1916592559 wait complete. 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.292+0000 7ff040bbc740 1 -- start start 2024-01-25T16:09:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 Processor -- start 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fe5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fe60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9f001e0 con 0x5608d9fdf9b0 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9effe80 con 0x5608d9fdf090 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9ec50c0 con 0x5608d9fe8780 2024-01-25T16:09:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff040bbc740 10 client.0 _ll_register_callbacks cb 0x5608d9eeaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34662/0 (socket says 172.21.15.33:34662) 2024-01-25T16:09:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 -- 172.21.15.33:0/3705952039 learned_addr learned my addr 172.21.15.33:0/3705952039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fe5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff037fff640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fe60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 -- 172.21.15.33:0/3705952039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 msgr2=0x5608d9fe60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fe60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 -- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 msgr2=0x5608d9fe5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fe5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.293+0000 7ff036ffd640 1 -- 172.21.15.33:0/3705952039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff030014d70 con 0x5608d9fdf9b0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff036ffd640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00bad0 tx=0x7ff02c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e36d17fe9230a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c016070 con 0x5608d9fdf9b0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608d9f40040 con 0x5608d9fdf9b0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff02c011440 con 0x5608d9fdf9b0 2024-01-25T16:09:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c003e80 con 0x5608d9fdf9b0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 10 client.15504 fetch_fsmap learned FSMap version 23 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608d9f456d0 con 0x5608d9fdf9b0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02c027080 con 0x5608d9fdf9b0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff02c01b7d0 con 0x5608d9fdf9b0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 10 client.15504 fetch_fsmap finished waiting for FSMap version 23 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 10 client.15504 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 20 client.15504 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608d9ff8600 0x5608d9ffaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 4 client.15504 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.294+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608d9f3fd60 con 0x5608d9ff8600 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.295+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608d9ff8600 0x5608d9ffaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.295+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608d9ff8600 0x5608d9ffaa20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=276f562cdf485a1c server_cookie=567d462d9b20f0f7 in_seq=0 out_seq=0 2024-01-25T16:09:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.295+0000 7ff016ffd640 10 client.15504 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff016ffd640 1 -- 172.21.15.33:0/3705952039 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608d9f3fd60 con 0x5608d9ff8600 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff016ffd640 10 client.15504 handle_command_reply: tid=0 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 client.15504 shutdown 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 msgr2=0x5608d9fe5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff02c00bad0 tx=0x7ff02c00bf90 comp rx=0 tx=0).stop 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 shutdown_connections 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608d9ff8600 0x5608d9ffaa20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fe60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5b90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff016ffd640 0 client.15504 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3705952039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fe5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 >> 172.21.15.33:0/3705952039 conn(0x5608d9fd99b0 msgr2=0x5608d9fdb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 shutdown_connections 2024-01-25T16:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 1 -- 172.21.15.33:0/3705952039 wait complete. 2024-01-25T16:09:11.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.296+0000 7ff040bbc740 20 client.15504 trim_cache size 0 max 16384 2024-01-25T16:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 Processor -- start 2024-01-25T16:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 -- start start 2024-01-25T16:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9fe05e0 con 0x5608d9fdf090 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9fda9b0 con 0x5608d9fdf9b0 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff040bbc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608d9fd94e0 con 0x5608d9fe8780 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34676/0 (socket says 172.21.15.33:34676) 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 -- 172.21.15.33:0/3724791348 learned_addr learned my addr 172.21.15.33:0/3724791348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff037fff640 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 -- 172.21.15.33:0/3724791348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 msgr2=0x5608d9fff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 -- 172.21.15.33:0/3724791348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf9b0 msgr2=0x5608d9fe5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 -- 172.21.15.33:0/3724791348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608d9ee2060 con 0x5608d9fdf090 2024-01-25T16:09:11.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff037fff640 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.310+0000 7ff0377fe640 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0280b8b90 tx=0x7ff0280efb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26118a03fab29b81 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff01d7fa640 1 -- 172.21.15.33:0/3724791348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0280bcd90 con 0x5608d9fdf090 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff01d7fa640 1 -- 172.21.15.33:0/3724791348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff02808cf80 con 0x5608d9fdf090 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff01d7fa640 1 -- 172.21.15.33:0/3724791348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0280ba860 con 0x5608d9fdf090 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 -- 172.21.15.33:0/3724791348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 msgr2=0x5608d9fe7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff0280b8b90 tx=0x7ff0280efb80 comp rx=0 tx=0).stop 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 -- 172.21.15.33:0/3724791348 shutdown_connections 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9fff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf9b0 0x5608d9fe5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 --2- 172.21.15.33:0/3724791348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf090 0x5608d9fe7d80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 -- 172.21.15.33:0/3724791348 >> 172.21.15.33:0/3724791348 conn(0x5608d9fd99b0 msgr2=0x5608d9fd9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 -- 172.21.15.33:0/3724791348 shutdown_connections 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.311+0000 7ff040bbc740 1 -- 172.21.15.33:0/3724791348 wait complete. 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 -- start start 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 Processor -- start 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fedff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9feea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da04eb30 con 0x5608d9fdf9b0 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da04ec70 con 0x5608d9fdf090 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608da04edb0 con 0x5608d9fe8780 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff040bbc740 10 client.0 _ll_register_callbacks cb 0x7ff030014850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34684/0 (socket says 172.21.15.33:34684) 2024-01-25T16:09:11.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff036ffd640 1 -- 172.21.15.33:0/699166736 learned_addr learned my addr 172.21.15.33:0/699166736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff0377fe640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fedff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.312+0000 7ff037fff640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9feea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 -- 172.21.15.33:0/699166736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 msgr2=0x5608d9feea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9feea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 -- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 msgr2=0x5608d9fedff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fedff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 -- 172.21.15.33:0/699166736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0381a3ee0 con 0x5608d9fdf9b0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff036ffd640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5608d9fe4eb0 tx=0x7ff02c01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c637c753d722b472 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c02a070 con 0x5608d9fdf9b0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ff02c000e40 con 0x5608d9fdf9b0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff03001a040 con 0x5608d9fdf9b0 2024-01-25T16:09:11.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff02c016040 con 0x5608d9fdf9b0 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 10 client.15510 fetch_fsmap learned FSMap version 23 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608d9f07a80 con 0x5608d9fdf9b0 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff03001a040 con 0x5608d9fdf9b0 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 23) v1 ==== 7886+0+0 (secure 0 0 0) 0x7ff02c02d030 con 0x5608d9fdf9b0 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 10 client.15510 fetch_fsmap finished waiting for FSMap version 23 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 10 client.15510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:11.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 20 client.15510 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608da0504c0 0x5608da0528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 4 client.15510 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.313+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5608d9c46600 con 0x5608da0504c0 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.314+0000 7ff0377fe640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608da0504c0 0x5608da0528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.314+0000 7ff016ffd640 10 client.15510 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:11.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.314+0000 7ff0377fe640 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608da0504c0 0x5608da0528e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdaff5f55fbd8fd2 server_cookie=b3f53dd7038baaee in_seq=0 out_seq=0 2024-01-25T16:09:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff016ffd640 1 -- 172.21.15.33:0/699166736 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5608d9c46600 con 0x5608da0504c0 2024-01-25T16:09:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff016ffd640 10 client.15510 handle_command_reply: tid=0 2024-01-25T16:09:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff040bbc740 1 client.15510 shutdown 2024-01-25T16:09:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 msgr2=0x5608d9fee530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5608d9fe4eb0 tx=0x7ff02c01b9f0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 shutdown_connections 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.376+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5608da0504c0 0x5608da0528e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5608d9fe8780 0x5608d9feea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5608d9fdf9b0 0x5608d9fee530 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 --2- 172.21.15.33:0/699166736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5608d9fdf090 0x5608d9fedff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff016ffd640 0 client.15510 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 >> 172.21.15.33:0/699166736 conn(0x5608d9fd99b0 msgr2=0x5608d9fff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 shutdown_connections 2024-01-25T16:09:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 1 -- 172.21.15.33:0/699166736 wait complete. 2024-01-25T16:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff040bbc740 20 client.15510 trim_cache size 0 max 16384 2024-01-25T16:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:09:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:09:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:09:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 msgr2=0x7ff0200a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 0x7ff0200a3ad0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff030014d40 tx=0x7ff030009760 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 msgr2=0x7ff038197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff028006010 tx=0x7ff028004dd0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 shutdown_connections 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff0200a1610 0x7ff0200a3ad0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff03810b340 0x7ff03819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff0381027f0 0x7ff0381976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 --2- 172.21.15.33:0/884549460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff038101ed0 0x7ff038197190 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 >> 172.21.15.33:0/884549460 conn(0x7ff038076d20 msgr2=0x7ff0380fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 shutdown_connections 2024-01-25T16:09:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:11.377+0000 7ff03dff5640 1 -- 172.21.15.33:0/884549460 wait complete. 2024-01-25T16:09:11.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:11 smithi033 ceph-mon[31417]: pgmap v43: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 354 KiB/s wr, 61 op/s 2024-01-25T16:09:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:11 smithi152 ceph-mon[36015]: pgmap v43: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 354 KiB/s wr, 61 op/s 2024-01-25T16:09:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:11 smithi076 ceph-mon[35642]: pgmap v43: 97 pgs: 97 active+clean; 4.8 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 354 KiB/s wr, 61 op/s 2024-01-25T16:09:11.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13042 10.67 MB/sec warmup 8 sec latency 1251.294 ms 2024-01-25T16:09:12.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13042 9.48 MB/sec warmup 9 sec latency 2251.385 ms 2024-01-25T16:09:13.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:13 smithi033 ceph-mon[31417]: pgmap v44: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 687 KiB/s wr, 100 op/s 2024-01-25T16:09:13.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:13 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:13 smithi152 ceph-mon[36015]: pgmap v44: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 687 KiB/s wr, 100 op/s 2024-01-25T16:09:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:13 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:13 smithi076 ceph-mon[35642]: pgmap v44: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 687 KiB/s wr, 100 op/s 2024-01-25T16:09:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:13 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:09:13.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13042 8.53 MB/sec warmup 10 sec latency 3251.487 ms 2024-01-25T16:09:14.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13042 7.76 MB/sec warmup 11 sec latency 4251.588 ms 2024-01-25T16:09:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:15 smithi033 ceph-mon[31417]: pgmap v45: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:15.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:15 smithi152 ceph-mon[36015]: pgmap v45: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:15 smithi076 ceph-mon[35642]: pgmap v45: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:15.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13143 7.17 MB/sec warmup 12 sec latency 4928.718 ms 2024-01-25T16:09:16.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:16 smithi033 ceph-mon[31417]: pgmap v46: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:16.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:16 smithi152 ceph-mon[36015]: pgmap v46: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:16.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:16 smithi076 ceph-mon[35642]: pgmap v46: 97 pgs: 97 active+clean; 8.1 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 665 KiB/s wr, 83 op/s 2024-01-25T16:09:16.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13143 6.62 MB/sec warmup 13 sec latency 1214.164 ms 2024-01-25T16:09:17.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13143 6.15 MB/sec warmup 14 sec latency 2214.249 ms 2024-01-25T16:09:18.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13143 5.74 MB/sec warmup 15 sec latency 3214.325 ms 2024-01-25T16:09:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:19 smithi076 ceph-mon[35642]: pgmap v47: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 704 KiB/s wr, 136 op/s 2024-01-25T16:09:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:19 smithi033 ceph-mon[31417]: pgmap v47: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 704 KiB/s wr, 136 op/s 2024-01-25T16:09:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:19 smithi152 ceph-mon[36015]: pgmap v47: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 704 KiB/s wr, 136 op/s 2024-01-25T16:09:19.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 13143 5.38 MB/sec warmup 16 sec latency 4214.478 ms 2024-01-25T16:09:20.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 14431 5.71 MB/sec warmup 17 sec latency 4794.244 ms 2024-01-25T16:09:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:21 smithi033 ceph-mon[31417]: pgmap v48: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 372 KiB/s wr, 92 op/s 2024-01-25T16:09:21.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:21 smithi152 ceph-mon[36015]: pgmap v48: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 372 KiB/s wr, 92 op/s 2024-01-25T16:09:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:21 smithi076 ceph-mon[35642]: pgmap v48: 97 pgs: 97 active+clean; 7.9 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 372 KiB/s wr, 92 op/s 2024-01-25T16:09:21.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16624 5.81 MB/sec warmup 18 sec latency 145.588 ms 2024-01-25T16:09:22.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:22 smithi033 ceph-mon[31417]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-25T16:09:22.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:22 smithi152 ceph-mon[36015]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-25T16:09:22.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:22 smithi076 ceph-mon[35642]: mgrmap e26: x(active, since 91s), standbys: y, z 2024-01-25T16:09:22.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16624 5.50 MB/sec warmup 19 sec latency 1145.683 ms 2024-01-25T16:09:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:23 smithi033 ceph-mon[31417]: pgmap v49: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 443 KiB/s wr, 109 op/s 2024-01-25T16:09:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:23 smithi152 ceph-mon[36015]: pgmap v49: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 443 KiB/s wr, 109 op/s 2024-01-25T16:09:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:23 smithi076 ceph-mon[35642]: pgmap v49: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 443 KiB/s wr, 109 op/s 2024-01-25T16:09:23.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16624 5.23 MB/sec warmup 20 sec latency 2145.780 ms 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- 172.21.15.33:0/2018197984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 msgr2=0x7f22f410dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/2018197984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f410dbb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f22ec009cd0 tx=0x7f22ec02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- 172.21.15.33:0/2018197984 shutdown_connections 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/2018197984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f410dbb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/2018197984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f41018c0 0x7f22f4105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/2018197984 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f41012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- 172.21.15.33:0/2018197984 >> 172.21.15.33:0/2018197984 conn(0x7f22f40fc770 msgr2=0x7f22f40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- 172.21.15.33:0/2018197984 shutdown_connections 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- 172.21.15.33:0/2018197984 wait complete. 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 Processor -- start 2024-01-25T16:09:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.473+0000 7f22fa4cf640 1 -- start start 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f4072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f4073a70 0x7f22f41a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a5880 con 0x7f22f4105fe0 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a59f0 con 0x7f22f4073a70 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22fa4cf640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41a5bb0 con 0x7f22f4100f10 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52036/0 (socket says 172.21.15.33:52036) 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 -- 172.21.15.33:0/3622526921 learned_addr learned my addr 172.21.15.33:0/3622526921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f4072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f4073a70 0x7f22f41a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 -- 172.21.15.33:0/3622526921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 msgr2=0x7f22f4072ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f4072ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 -- 172.21.15.33:0/3622526921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f4073a70 msgr2=0x7f22f41a52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f4073a70 0x7f22f41a52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 -- 172.21.15.33:0/3622526921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec009900 con 0x7f22f4105fe0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f4072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.474+0000 7f22f37fe640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c970 tx=0x7f22e800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.475+0000 7f22f17fa640 1 -- 172.21.15.33:0/3622526921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8007bb0 con 0x7f22f4105fe0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.475+0000 7f22f17fa640 1 -- 172.21.15.33:0/3622526921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e8007d10 con 0x7f22f4105fe0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.475+0000 7f22f17fa640 1 -- 172.21.15.33:0/3622526921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8004e20 con 0x7f22f4105fe0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.475+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f41a5e90 con 0x7f22f4105fe0 2024-01-25T16:09:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.475+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f41a62c0 con 0x7f22f4105fe0 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.477+0000 7f22f17fa640 1 -- 172.21.15.33:0/3622526921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f22e8004f80 con 0x7f22f4105fe0 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.477+0000 7f22fd16c740 1 Processor -- start 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.477+0000 7f22fd16c740 1 -- start start 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d5684090 0x5607d5684470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f17fa640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 0x7f22dc0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d55f3070 con 0x5607d568d780 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d55f0760 con 0x5607d5684090 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22fd16c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d55ec050 con 0x5607d56849b0 2024-01-25T16:09:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f37fe640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 0x7f22dc0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52040/0 (socket says 172.21.15.33:52040) 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 -- 172.21.15.33:0/925808595 learned_addr learned my addr 172.21.15.33:0/925808595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f17fa640 1 -- 172.21.15.33:0/3622526921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22e80ccc60 con 0x7f22f4105fe0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f3fff640 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d5684090 0x5607d5684470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 -- 172.21.15.33:0/925808595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 msgr2=0x5607d568d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 -- 172.21.15.33:0/925808595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d5684090 msgr2=0x5607d5684470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d5684090 0x5607d5684470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 -- 172.21.15.33:0/925808595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607d55e3e70 con 0x5607d568d780 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f37fe640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 0x7f22dc0a39e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22e8099710 tx=0x7f22e808f180 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.478+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f22ec032ff0 tx=0x7f22ec009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62539cc935a18d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22da7fc640 1 -- 172.21.15.33:0/925808595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec034640 con 0x5607d568d780 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22da7fc640 1 -- 172.21.15.33:0/925808595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec003970 con 0x5607d568d780 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22da7fc640 1 -- 172.21.15.33:0/925808595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0042a0 con 0x5607d568d780 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 -- 172.21.15.33:0/925808595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 msgr2=0x5607d568fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f22ec032ff0 tx=0x7f22ec009360 comp rx=0 tx=0).stop 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 -- 172.21.15.33:0/925808595 shutdown_connections 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d568d780 0x5607d568fb70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 --2- 172.21.15.33:0/925808595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d5684090 0x5607d5684470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 -- 172.21.15.33:0/925808595 >> 172.21.15.33:0/925808595 conn(0x5607d567e9b0 msgr2=0x5607d567ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 -- 172.21.15.33:0/925808595 shutdown_connections 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.479+0000 7f22fd16c740 1 -- 172.21.15.33:0/925808595 wait complete. 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 -- start start 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 Processor -- start 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 0x5607d568a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d568b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d55a51e0 con 0x5607d56849b0 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d55a4e80 con 0x5607d568d780 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d556a0c0 con 0x5607d5684090 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22fd16c740 10 client.0 _ll_register_callbacks cb 0x5607d558fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22f3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22f3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52052/0 (socket says 172.21.15.33:52052) 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.480+0000 7f22f3fff640 1 -- 172.21.15.33:0/3111897214 learned_addr learned my addr 172.21.15.33:0/3111897214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 0x5607d568a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 -- 172.21.15.33:0/3111897214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 msgr2=0x5607d568a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 0x5607d568a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 -- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 msgr2=0x5607d568b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d568b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 -- 172.21.15.33:0/3111897214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f410e3c0 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 0x5607d568a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011a90 tx=0x7f22e4011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ad7d1e372d7ea0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400fe70 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22e4004050 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607d55e5040 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22e4004250 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22fd16c740 10 client.15522 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5607d55ea6d0 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.481+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22e4025080 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f22e4004a10 con 0x5607d56849b0 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 10 client.15522 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 20 client.15522 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d569d660 0x5607d569fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 4 client.15522 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607d55e4d60 con 0x5607d569d660 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d569d660 0x5607d569fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d569d660 0x5607d569fa80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6b465ad20658dd server_cookie=d48c24eda76b9fa9 in_seq=0 out_seq=0 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.482+0000 7f22d37fe640 10 client.15522 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22d37fe640 1 -- 172.21.15.33:0/3111897214 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607d55e4d60 con 0x5607d569d660 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22d37fe640 10 client.15522 handle_command_reply: tid=0 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 client.15522 shutdown 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 msgr2=0x5607d568ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f22e4011a90 tx=0x7f22e4011f50 comp rx=0 tx=0).stop 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 shutdown_connections 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d569d660 0x5607d569fa80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d568b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d56849b0 0x5607d568ac70 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22d37fe640 0 client.15522 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3111897214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d5684090 0x5607d568a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 >> 172.21.15.33:0/3111897214 conn(0x5607d567e9b0 msgr2=0x5607d5680700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 shutdown_connections 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 1 -- 172.21.15.33:0/3111897214 wait complete. 2024-01-25T16:09:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.483+0000 7f22fd16c740 20 client.15522 trim_cache size 0 max 16384 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 Processor -- start 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 -- start start 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d5694410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d569bd60 con 0x5607d5684090 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d56a4cb0 con 0x5607d568d780 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22fd16c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d5683840 con 0x5607d56849b0 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52066/0 (socket says 172.21.15.33:52066) 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 -- 172.21.15.33:0/3612791249 learned_addr learned my addr 172.21.15.33:0/3612791249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 -- 172.21.15.33:0/3612791249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 msgr2=0x5607d568afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 -- 172.21.15.33:0/3612791249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 msgr2=0x5607d5694410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d5694410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 -- 172.21.15.33:0/3612791249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607d5587060 con 0x5607d5684090 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f3fff640 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.497+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f22ec032630 tx=0x7f22ec009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c07f2752b5e94df server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22d9ffb640 1 -- 172.21.15.33:0/3612791249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec034760 con 0x5607d5684090 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22d9ffb640 1 -- 172.21.15.33:0/3612791249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec032690 con 0x5607d5684090 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22d9ffb640 1 -- 172.21.15.33:0/3612791249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec0568d0 con 0x5607d5684090 2024-01-25T16:09:24.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- 172.21.15.33:0/3612791249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 msgr2=0x5607d5680510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f22ec032630 tx=0x7f22ec009800 comp rx=0 tx=0).stop 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- 172.21.15.33:0/3612791249 shutdown_connections 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d568d780 0x5607d5694410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d56849b0 0x5607d568afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 --2- 172.21.15.33:0/3612791249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5680510 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- 172.21.15.33:0/3612791249 >> 172.21.15.33:0/3612791249 conn(0x5607d567e9b0 msgr2=0x5607d567fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- 172.21.15.33:0/3612791249 shutdown_connections 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- 172.21.15.33:0/3612791249 wait complete. 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.498+0000 7f22fd16c740 1 -- start start 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 Processor -- start 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d56849b0 0x5607d5693dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.507 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d568d780 0x5607d56ecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d56ed090 con 0x5607d5684090 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d56ed200 con 0x5607d56849b0 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607d56ed3c0 con 0x5607d568d780 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22fd16c740 10 client.0 _ll_register_callbacks cb 0x7f22ec037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52068/0 (socket says 172.21.15.33:52068) 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 -- 172.21.15.33:0/837436340 learned_addr learned my addr 172.21.15.33:0/837436340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f3fff640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d56849b0 0x5607d5693dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d568d780 0x5607d56ecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 -- 172.21.15.33:0/837436340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d568d780 msgr2=0x5607d56ecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d568d780 0x5607d56ecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 -- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d56849b0 msgr2=0x5607d5693dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d56849b0 0x5607d5693dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.499+0000 7f22f8a6c640 1 -- 172.21.15.33:0/837436340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607d55e3e70 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22f8a6c640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0093a0 tx=0x7f22ec02c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fe3660095094ec server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec034a70 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f22ec034bd0 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22ec003970 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ec034d90 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 10 client.15528 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5607d55aca80 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f22ec065020 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f22ec060070 con 0x5607d5684090 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 10 client.15528 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 10 client.15528 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 20 client.15528 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d56f5920 0x5607d56f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 4 client.15528 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.500+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5607d52eb600 con 0x5607d56f5920 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.501+0000 7f22f3fff640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d56f5920 0x5607d56f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.501+0000 7f22f3fff640 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d56f5920 0x5607d56f7d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8244b6a6d18aa12 server_cookie=8b3b9ea0c16a5641 in_seq=0 out_seq=0 2024-01-25T16:09:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.501+0000 7f22d37fe640 10 client.15528 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22d37fe640 1 -- 172.21.15.33:0/837436340 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5607d52eb600 con 0x5607d56f5920 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22d37fe640 10 client.15528 handle_command_reply: tid=0 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 client.15528 shutdown 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 msgr2=0x5607d5693880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f22ec0093a0 tx=0x7f22ec02c770 comp rx=0 tx=0).stop 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 shutdown_connections 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5607d56f5920 0x5607d56f7d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5607d568d780 0x5607d56ecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5607d56849b0 0x5607d5693dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22d37fe640 0 client.15528 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 --2- 172.21.15.33:0/837436340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607d5684090 0x5607d5693880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 >> 172.21.15.33:0/837436340 conn(0x5607d567e9b0 msgr2=0x5607d5680660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 shutdown_connections 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 1 -- 172.21.15.33:0/837436340 wait complete. 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.542+0000 7f22fd16c740 20 client.15528 trim_cache size 0 max 16384 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:09:24.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 msgr2=0x7f22dc0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 0x7f22dc0a39e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22e8099710 tx=0x7f22e808f180 comp rx=0 tx=0).stop 2024-01-25T16:09:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 msgr2=0x7f22f4073530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c970 tx=0x7f22e800ce30 comp rx=0 tx=0).stop 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 shutdown_connections 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f22dc0a1520 0x7f22dc0a39e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f4073a70 0x7f22f41a52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22f4105fe0 0x7f22f4073530 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 --2- 172.21.15.33:0/3622526921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22f4100f10 0x7f22f4072ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 >> 172.21.15.33:0/3622526921 conn(0x7f22f40fc770 msgr2=0x7f22f40fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 shutdown_connections 2024-01-25T16:09:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:24.543+0000 7f22fa4cf640 1 -- 172.21.15.33:0/3622526921 wait complete. 2024-01-25T16:09:24.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:24 smithi033 ceph-mon[31417]: pgmap v50: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:24.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:24 smithi152 ceph-mon[36015]: pgmap v50: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:24.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:24 smithi076 ceph-mon[35642]: pgmap v50: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:24.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16624 4.98 MB/sec warmup 21 sec latency 3145.873 ms 2024-01-25T16:09:25.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16685 4.78 MB/sec warmup 22 sec latency 3819.428 ms 2024-01-25T16:09:26.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16685 4.57 MB/sec warmup 23 sec latency 1266.212 ms 2024-01-25T16:09:27.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:27 smithi152 ceph-mon[36015]: pgmap v51: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:27 smithi076 ceph-mon[35642]: pgmap v51: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:27 smithi033 ceph-mon[31417]: pgmap v51: 97 pgs: 97 active+clean; 8.6 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 111 KiB/s wr, 69 op/s 2024-01-25T16:09:27.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16685 4.38 MB/sec warmup 24 sec latency 2266.305 ms 2024-01-25T16:09:28.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16685 4.21 MB/sec warmup 25 sec latency 3266.419 ms 2024-01-25T16:09:29.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:29 smithi152 ceph-mon[36015]: pgmap v52: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 221 KiB/s wr, 116 op/s 2024-01-25T16:09:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:29 smithi076 ceph-mon[35642]: pgmap v52: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 221 KiB/s wr, 116 op/s 2024-01-25T16:09:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:29 smithi033 ceph-mon[31417]: pgmap v52: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 221 KiB/s wr, 116 op/s 2024-01-25T16:09:29.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 16685 4.05 MB/sec warmup 26 sec latency 4266.472 ms 2024-01-25T16:09:30.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 18454 4.33 MB/sec warmup 27 sec latency 4473.427 ms 2024-01-25T16:09:31.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:31 smithi152 ceph-mon[36015]: pgmap v53: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 181 KiB/s wr, 62 op/s 2024-01-25T16:09:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:31 smithi076 ceph-mon[35642]: pgmap v53: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 181 KiB/s wr, 62 op/s 2024-01-25T16:09:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:31 smithi033 ceph-mon[31417]: pgmap v53: 97 pgs: 97 active+clean; 5.6 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 181 KiB/s wr, 62 op/s 2024-01-25T16:09:31.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20173 4.41 MB/sec warmup 28 sec latency 425.104 ms 2024-01-25T16:09:32.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20173 4.26 MB/sec warmup 29 sec latency 1425.226 ms 2024-01-25T16:09:33.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:33 smithi152 ceph-mon[36015]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-01-25T16:09:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:33 smithi076 ceph-mon[35642]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-01-25T16:09:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:33 smithi033 ceph-mon[31417]: pgmap v54: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 3.0 MiB/s wr, 132 op/s 2024-01-25T16:09:33.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20173 4.12 MB/sec warmup 30 sec latency 2425.329 ms 2024-01-25T16:09:34.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20173 3.99 MB/sec warmup 31 sec latency 3425.383 ms 2024-01-25T16:09:35.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:35 smithi152 ceph-mon[36015]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:35 smithi076 ceph-mon[35642]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:35 smithi033 ceph-mon[31417]: pgmap v55: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:35.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20274 3.89 MB/sec warmup 32 sec latency 4098.796 ms 2024-01-25T16:09:36.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20274 3.77 MB/sec warmup 33 sec latency 1221.758 ms 2024-01-25T16:09:37.383 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:37 smithi152 ceph-mon[36015]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:37.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:37 smithi076 ceph-mon[35642]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:37 smithi033 ceph-mon[31417]: pgmap v56: 97 pgs: 97 active+clean; 25 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.9 MiB/s wr, 116 op/s 2024-01-25T16:09:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 -- 172.21.15.33:0/1059417594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fff60 msgr2=0x7f3a20103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 --2- 172.21.15.33:0/1059417594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fff60 0x7f3a20103fa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009930 tx=0x7f3a1002bc30 comp rx=0 tx=0).stop 2024-01-25T16:09:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 -- 172.21.15.33:0/1059417594 shutdown_connections 2024-01-25T16:09:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 --2- 172.21.15.33:0/1059417594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 0x7f3a2010c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 --2- 172.21.15.33:0/1059417594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fff60 0x7f3a20103fa0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 --2- 172.21.15.33:0/1059417594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 -- 172.21.15.33:0/1059417594 >> 172.21.15.33:0/1059417594 conn(0x7f3a200761b0 msgr2=0x7f3a200fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 -- 172.21.15.33:0/1059417594 shutdown_connections 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.641+0000 7f3a28275640 1 -- 172.21.15.33:0/1059417594 wait complete. 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 Processor -- start 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 -- start start 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 0x7f3a200fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201a5990 con 0x7f3a200fc040 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201a5b00 con 0x7f3a20104680 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a28275640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201a5cc0 con 0x7f3a200ff5b0 2024-01-25T16:09:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47636/0 (socket says 172.21.15.33:47636) 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a26812640 1 -- 172.21.15.33:0/337994417 learned_addr learned my addr 172.21.15.33:0/337994417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a26011640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a25810640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 0x7f3a200fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.642+0000 7f3a26812640 1 -- 172.21.15.33:0/337994417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 msgr2=0x7f3a200fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26812640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26812640 1 -- 172.21.15.33:0/337994417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 msgr2=0x7f3a200fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26812640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 0x7f3a200fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26812640 1 -- 172.21.15.33:0/337994417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009560 con 0x7f3a200fc040 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26011640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a26812640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800de10 tx=0x7f3a1800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a177fe640 1 -- 172.21.15.33:0/337994417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18003b70 con 0x7f3a200fc040 2024-01-25T16:09:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a177fe640 1 -- 172.21.15.33:0/337994417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1800a950 con 0x7f3a200fc040 2024-01-25T16:09:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a201a5fa0 con 0x7f3a200fc040 2024-01-25T16:09:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a177fe640 1 -- 172.21.15.33:0/337994417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800e9d0 con 0x7f3a200fc040 2024-01-25T16:09:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.643+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201a6430 con 0x7f3a200fc040 2024-01-25T16:09:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.645+0000 7f3a177fe640 1 -- 172.21.15.33:0/337994417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3a1800eb30 con 0x7f3a200fc040 2024-01-25T16:09:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 Processor -- start 2024-01-25T16:09:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 -- start start 2024-01-25T16:09:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a177fe640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 0x7f3a0c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fd6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71f45070 con 0x555c71fdf780 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71f42760 con 0x555c71fd69b0 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71f3e050 con 0x555c71fd6090 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a25810640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 0x7f3a0c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47652/0 (socket says 172.21.15.33:47652) 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a26812640 1 -- 172.21.15.33:0/4259492240 learned_addr learned my addr 172.21.15.33:0/4259492240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a177fe640 1 -- 172.21.15.33:0/337994417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a180cd190 con 0x7f3a200fc040 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a26011640 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fd6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.646+0000 7f3a25810640 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 -- 172.21.15.33:0/4259492240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 msgr2=0x555c71fd6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fd6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 -- 172.21.15.33:0/4259492240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 msgr2=0x555c71fdf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 -- 172.21.15.33:0/4259492240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c71f35e70 con 0x555c71fdf780 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a25810640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 0x7f3a0c0a39e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a200fd050 tx=0x7f3a10004740 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a26812640 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3a18096ac0 tx=0x7f3a1800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658a0bf37e06bf49 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a14bfe640 1 -- 172.21.15.33:0/4259492240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180999b0 con 0x555c71fdf780 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a14bfe640 1 -- 172.21.15.33:0/4259492240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a18015c30 con 0x555c71fdf780 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.647+0000 7f3a14bfe640 1 -- 172.21.15.33:0/4259492240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18096760 con 0x555c71fdf780 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4259492240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 msgr2=0x555c71fe1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3a18096ac0 tx=0x7f3a1800bf90 comp rx=0 tx=0).stop 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4259492240 shutdown_connections 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fe1b70 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4259492240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fd6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4259492240 >> 172.21.15.33:0/4259492240 conn(0x555c71fd09b0 msgr2=0x555c71fd0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4259492240 shutdown_connections 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4259492240 wait complete. 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 -- start start 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.648+0000 7f3a2ae3a740 1 Processor -- start 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fdcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c71fdd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71ef71e0 con 0x555c71fd6090 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71ef6e80 con 0x555c71fdf780 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71ebc0c0 con 0x555c71fd69b0 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a2ae3a740 10 client.0 _ll_register_callbacks cb 0x555c71ee1ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:37.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47660/0 (socket says 172.21.15.33:47660) 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 -- 172.21.15.33:0/4207128083 learned_addr learned my addr 172.21.15.33:0/4207128083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a25810640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fdcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26812640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c71fdd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 -- 172.21.15.33:0/4207128083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 msgr2=0x555c71fdcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fdcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 -- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 msgr2=0x555c71fdd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c71fdd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 -- 172.21.15.33:0/4207128083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201a6430 con 0x555c71fd6090 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a26011640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00bf10 tx=0x7f3a1c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f48f37a7fafcb4e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c004500 con 0x555c71fd6090 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.649+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a1c0035f0 con 0x555c71fd6090 2024-01-25T16:09:37.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c71f37040 con 0x555c71fd6090 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1c0037b0 con 0x555c71fd6090 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 10 client.15540 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c71f3c6d0 con 0x555c71fd6090 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1c027080 con 0x555c71fd6090 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3a1c010b40 con 0x555c71fd6090 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 10 client.15540 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 10 client.15540 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 20 client.15540 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c71fef700 0x555c71ff1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 4 client.15540 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c71f36d60 con 0x555c71fef700 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.650+0000 7f3a25810640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c71fef700 0x555c71ff1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.651+0000 7f3a09ffb640 10 client.15540 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:37.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.651+0000 7f3a25810640 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c71fef700 0x555c71ff1b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf3e1b5ab00c68e server_cookie=24902b3fd7cea7a8 in_seq=0 out_seq=0 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.651+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4207128083 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c71f36d60 con 0x555c71fef700 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.651+0000 7f3a09ffb640 10 client.15540 handle_command_reply: tid=0 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.651+0000 7f3a2ae3a740 1 client.15540 shutdown 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 msgr2=0x555c71fdc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00bf10 tx=0x7f3a1c012820 comp rx=0 tx=0).stop 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 shutdown_connections 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c71fef700 0x555c71ff1b20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c71fdd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fdcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a09ffb640 0 client.15540 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4207128083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fdc6c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 >> 172.21.15.33:0/4207128083 conn(0x555c71fd09b0 msgr2=0x555c71fd2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 shutdown_connections 2024-01-25T16:09:37.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4207128083 wait complete. 2024-01-25T16:09:37.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.652+0000 7f3a2ae3a740 20 client.15540 trim_cache size 0 max 16384 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 Processor -- start 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 -- start start 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fdedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71fd5ca0 con 0x555c71fdf780 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71fedd60 con 0x555c71fd69b0 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c71fdda10 con 0x555c71fd6090 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a26812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47672/0 (socket says 172.21.15.33:47672) 2024-01-25T16:09:37.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.665+0000 7f3a26812640 1 -- 172.21.15.33:0/3540268972 learned_addr learned my addr 172.21.15.33:0/3540268972 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a25810640 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26011640 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fdedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 -- 172.21.15.33:0/3540268972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 msgr2=0x555c71fdedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fdedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 -- 172.21.15.33:0/3540268972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 msgr2=0x555c71fdcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 -- 172.21.15.33:0/3540268972 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c71ed9060 con 0x555c71fdf780 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a26812640 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3a18015da0 tx=0x7f3a18015e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680876d0e740fe5f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a0bfff640 1 -- 172.21.15.33:0/3540268972 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a180999b0 con 0x555c71fdf780 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a0bfff640 1 -- 172.21.15.33:0/3540268972 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a1809a750 con 0x555c71fdf780 2024-01-25T16:09:37.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.666+0000 7f3a0bfff640 1 -- 172.21.15.33:0/3540268972 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1809a510 con 0x555c71fdf780 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/3540268972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 msgr2=0x555c71fd9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3a18015da0 tx=0x7f3a18015e70 comp rx=0 tx=0).stop 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/3540268972 shutdown_connections 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fdf780 0x555c71fd9940 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fd69b0 0x555c71fdcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/3540268972 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd6090 0x555c71fdedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/3540268972 >> 172.21.15.33:0/3540268972 conn(0x555c71fd09b0 msgr2=0x555c71fd79c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/3540268972 shutdown_connections 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/3540268972 wait complete. 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- start start 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 Processor -- start 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.674 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fe4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c7203ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7203f110 con 0x555c71fd6090 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7203f250 con 0x555c71fdf780 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c7203f410 con 0x555c71fd69b0 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.667+0000 7f3a2ae3a740 10 client.0 _ll_register_callbacks cb 0x7f3a180045f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47686/0 (socket says 172.21.15.33:47686) 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 -- 172.21.15.33:0/4200267170 learned_addr learned my addr 172.21.15.33:0/4200267170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 -- 172.21.15.33:0/4200267170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 msgr2=0x555c71fe4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a25810640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fe4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26812640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c7203ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fe4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 -- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 msgr2=0x555c7203ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:37.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c7203ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 -- 172.21.15.33:0/4200267170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c71f35e70 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a25810640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fe4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a26011640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534e1fcf8eddbc87 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c00bb00 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3a1c004880 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a18015c30 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1c00b7c0 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a2ae3a740 10 client.15546 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x555c71efea80 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.668+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3a18015c30 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3a1c035070 con 0x555c71fd6090 2024-01-25T16:09:37.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 10 client.15546 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 10 client.15546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 20 client.15546 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c72046ce0 0x555c72049100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 4 client.15546 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555c71c3d600 con 0x555c72046ce0 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a25810640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c72046ce0 0x555c72049100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a09ffb640 10 client.15546 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:37.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.669+0000 7f3a25810640 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c72046ce0 0x555c72049100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ff0d1f4d70f6f03 server_cookie=12001799c01af4a2 in_seq=0 out_seq=0 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a09ffb640 1 -- 172.21.15.33:0/4200267170 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555c71c3d600 con 0x555c72046ce0 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a09ffb640 10 client.15546 handle_command_reply: tid=0 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 client.15546 shutdown 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 msgr2=0x555c71fe4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c006010 tx=0x7f3a1c0052f0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 shutdown_connections 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555c72046ce0 0x555c72049100 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555c71fdf780 0x555c7203ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555c71fd69b0 0x555c71fe4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a09ffb640 0 client.15546 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 --2- 172.21.15.33:0/4200267170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c71fd6090 0x555c71fe4710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 >> 172.21.15.33:0/4200267170 conn(0x555c71fd09b0 msgr2=0x555c71fdcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 shutdown_connections 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 1 -- 172.21.15.33:0/4200267170 wait complete. 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.705+0000 7f3a2ae3a740 20 client.15546 trim_cache size 0 max 16384 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:09:37.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 msgr2=0x7f3a0c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 0x7f3a0c0a39e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3a200fd050 tx=0x7f3a10004740 comp rx=0 tx=0).stop 2024-01-25T16:09:37.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 msgr2=0x7f3a201a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800de10 tx=0x7f3a1800b730 comp rx=0 tx=0).stop 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 shutdown_connections 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3a0c0a1520 0x7f3a0c0a39e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a200fc040 0x7f3a201a53c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a20104680 0x7f3a200fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 --2- 172.21.15.33:0/337994417 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3a200ff5b0 0x7f3a200fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 >> 172.21.15.33:0/337994417 conn(0x7f3a200761b0 msgr2=0x7f3a20102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 shutdown_connections 2024-01-25T16:09:37.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:37.706+0000 7f3a28275640 1 -- 172.21.15.33:0/337994417 wait complete. 2024-01-25T16:09:37.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20274 3.66 MB/sec warmup 34 sec latency 2221.828 ms 2024-01-25T16:09:38.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20274 3.55 MB/sec warmup 35 sec latency 3221.925 ms 2024-01-25T16:09:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:39 smithi076 ceph-mon[35642]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.0 MiB/s wr, 127 op/s 2024-01-25T16:09:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:39 smithi033 ceph-mon[31417]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.0 MiB/s wr, 127 op/s 2024-01-25T16:09:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:39 smithi152 ceph-mon[36015]: pgmap v57: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 3.0 MiB/s wr, 127 op/s 2024-01-25T16:09:39.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 20274 3.45 MB/sec warmup 36 sec latency 4222.041 ms 2024-01-25T16:09:40.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 21304 3.62 MB/sec warmup 37 sec latency 4896.799 ms 2024-01-25T16:09:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:41 smithi076 ceph-mon[35642]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-01-25T16:09:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:41 smithi033 ceph-mon[31417]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-01-25T16:09:41.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:41 smithi152 ceph-mon[36015]: pgmap v58: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.9 MiB/s wr, 80 op/s 2024-01-25T16:09:41.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23660 3.75 MB/sec warmup 38 sec latency 51.104 ms 2024-01-25T16:09:42.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23660 3.66 MB/sec warmup 39 sec latency 1051.187 ms 2024-01-25T16:09:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:43 smithi076 ceph-mon[35642]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.9 MiB/s wr, 109 op/s 2024-01-25T16:09:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:43 smithi033 ceph-mon[31417]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.9 MiB/s wr, 109 op/s 2024-01-25T16:09:43.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:43 smithi152 ceph-mon[36015]: pgmap v59: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 2.9 MiB/s wr, 109 op/s 2024-01-25T16:09:43.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23660 3.56 MB/sec warmup 40 sec latency 2051.278 ms 2024-01-25T16:09:44.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23660 3.48 MB/sec warmup 41 sec latency 3051.393 ms 2024-01-25T16:09:45.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:45 smithi076 ceph-mon[35642]: pgmap v60: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:45 smithi033 ceph-mon[31417]: pgmap v60: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:45 smithi152 ceph-mon[36015]: pgmap v60: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:45.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23714 3.40 MB/sec warmup 42 sec latency 3737.581 ms 2024-01-25T16:09:46.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23714 3.32 MB/sec warmup 43 sec latency 1278.007 ms 2024-01-25T16:09:47.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:47 smithi076 ceph-mon[35642]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:47 smithi033 ceph-mon[31417]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:47 smithi152 ceph-mon[36015]: pgmap v61: 97 pgs: 97 active+clean; 26 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 117 KiB/s wr, 40 op/s 2024-01-25T16:09:47.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23714 3.24 MB/sec warmup 44 sec latency 2278.107 ms 2024-01-25T16:09:48.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23714 3.17 MB/sec warmup 45 sec latency 3278.179 ms 2024-01-25T16:09:49.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:49 smithi076 ceph-mon[35642]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 238 KiB/s wr, 57 op/s 2024-01-25T16:09:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:49 smithi033 ceph-mon[31417]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 238 KiB/s wr, 57 op/s 2024-01-25T16:09:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:49 smithi152 ceph-mon[36015]: pgmap v62: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 238 KiB/s wr, 57 op/s 2024-01-25T16:09:49.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23714 3.10 MB/sec warmup 46 sec latency 4278.301 ms 2024-01-25T16:09:50.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23754 3.04 MB/sec warmup 47 sec latency 4951.925 ms 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/866817652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc068370 msgr2=0x7f7abc0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/866817652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc068370 0x7f7abc0687d0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009900 tx=0x7f7ab002bcd0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/866817652 shutdown_connections 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/866817652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 0x7f7abc10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/866817652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc068370 0x7f7abc0687d0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/866817652 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc106de0 0x7f7abc1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/866817652 >> 172.21.15.33:0/866817652 conn(0x7f7abc076a40 msgr2=0x7f7abc076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/866817652 shutdown_connections 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/866817652 wait complete. 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.803+0000 7f7ac1e2b640 1 Processor -- start 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 -- start start 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc068370 0x7f7abc1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 0x7f7abc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19bf60 con 0x7f7abc106de0 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19c0d0 con 0x7f7abc068d10 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7ac1e2b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7abc19c290 con 0x7f7abc068370 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44772/0 (socket says 172.21.15.33:44772) 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 -- 172.21.15.33:0/2479703629 learned_addr learned my addr 172.21.15.33:0/2479703629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc068370 0x7f7abc1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abaffd640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 0x7f7abc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 -- 172.21.15.33:0/2479703629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc068370 msgr2=0x7f7abc1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc068370 0x7f7abc1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 -- 172.21.15.33:0/2479703629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 msgr2=0x7f7abc1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 0x7f7abc1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.804+0000 7f7abbfff640 1 -- 172.21.15.33:0/2479703629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab0009560 con 0x7f7abc106de0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400cab0 tx=0x7f7ab400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7ab8ff9640 1 -- 172.21.15.33:0/2479703629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab400ae20 con 0x7f7abc106de0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7ab8ff9640 1 -- 172.21.15.33:0/2479703629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab400f910 con 0x7f7abc106de0 2024-01-25T16:09:50.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7abc19c570 con 0x7f7abc106de0 2024-01-25T16:09:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7ab8ff9640 1 -- 172.21.15.33:0/2479703629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40189a0 con 0x7f7abc106de0 2024-01-25T16:09:50.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.805+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a3df0 con 0x7f7abc106de0 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.807+0000 7f7ab8ff9640 1 -- 172.21.15.33:0/2479703629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7ab400fa80 con 0x7f7abc106de0 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.807+0000 7f7ac49f5740 1 Processor -- start 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.807+0000 7f7ac49f5740 1 -- start start 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.807+0000 7f7ab8ff9640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 0x7f7aa40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce58090 0x55d39ce58470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce589b0 0x55d39ce61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cdc7070 con 0x55d39ce61780 2024-01-25T16:09:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cdc4760 con 0x55d39ce58090 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cdc0050 con 0x55d39ce589b0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7ab8ff9640 1 -- 172.21.15.33:0/2479703629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7ab40cd3d0 con 0x7f7abc106de0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abaffd640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 0x7f7aa40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44782/0 (socket says 172.21.15.33:44782) 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 -- 172.21.15.33:0/2894758802 learned_addr learned my addr 172.21.15.33:0/2894758802 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce58090 0x55d39ce58470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abaffd640 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce589b0 0x55d39ce61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 -- 172.21.15.33:0/2894758802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce589b0 msgr2=0x55d39ce61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce589b0 0x55d39ce61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 -- 172.21.15.33:0/2894758802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce58090 msgr2=0x55d39ce58470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce58090 0x55d39ce58470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 -- 172.21.15.33:0/2894758802 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d39cdb7e70 con 0x55d39ce61780 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4096c50 tx=0x7f7ab4020da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaf791cc1b5a611 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7abaffd640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 0x7f7aa40a39b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7abc076fd0 tx=0x7f7ab0004a30 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7aa1ffb640 1 -- 172.21.15.33:0/2894758802 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab409e120 con 0x55d39ce61780 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7aa1ffb640 1 -- 172.21.15.33:0/2894758802 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab409e330 con 0x55d39ce61780 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.808+0000 7f7aa1ffb640 1 -- 172.21.15.33:0/2894758802 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40cadb0 con 0x55d39ce61780 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2894758802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 msgr2=0x55d39ce63b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4096c50 tx=0x7f7ab4020da0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2894758802 shutdown_connections 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce63b70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce589b0 0x55d39ce61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2894758802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce58090 0x55d39ce58470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2894758802 >> 172.21.15.33:0/2894758802 conn(0x55d39ce529b0 msgr2=0x55d39ce52d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2894758802 shutdown_connections 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2894758802 wait complete. 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.809+0000 7f7ac49f5740 1 -- start start 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 Processor -- start 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce5ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce5f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cd791e0 con 0x55d39ce58090 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cd78e80 con 0x55d39ce589b0 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39cd3e0c0 con 0x55d39ce61780 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7ac49f5740 10 client.0 _ll_register_callbacks cb 0x55d39cd63ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44798/0 (socket says 172.21.15.33:44798) 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 -- 172.21.15.33:0/3833326189 learned_addr learned my addr 172.21.15.33:0/3833326189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abbfff640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce5f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abaffd640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce5ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 -- 172.21.15.33:0/3833326189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 msgr2=0x55d39ce5f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce5f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 -- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 msgr2=0x55d39ce5ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce5ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abb7fe640 1 -- 172.21.15.33:0/3833326189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7abc1a3df0 con 0x55d39ce58090 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abaffd640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce5ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.810+0000 7f7abbfff640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce5f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7aac012720 tx=0x7f7aac012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49425edb23bace0c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:50.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac017070 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7aac00f920 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d39cdb9040 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aac0122e0 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 10 client.15558 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d39cdbe6d0 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7aac028080 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7aac01c420 con 0x55d39ce58090 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 10 client.15558 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 20 client.15558 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39ce716d0 0x55d39ce73b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 4 client.15558 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.811+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d39cdb8d60 con 0x55d39ce716d0 2024-01-25T16:09:50.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.812+0000 7f7abaffd640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39ce716d0 0x55d39ce73b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.812+0000 7f7a9affd640 10 client.15558 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.812+0000 7f7abaffd640 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39ce716d0 0x55d39ce73b10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d37b68e34877961 server_cookie=aefb3615a14d6364 in_seq=0 out_seq=0 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7a9affd640 1 -- 172.21.15.33:0/3833326189 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d39cdb8d60 con 0x55d39ce716d0 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7a9affd640 10 client.15558 handle_command_reply: tid=0 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 client.15558 shutdown 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 msgr2=0x55d39ce5e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7aac012720 tx=0x7f7aac012be0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 shutdown_connections 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39ce716d0 0x55d39ce73b10 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce5f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7a9affd640 0 client.15558 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:50.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce5ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/3833326189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce5e730 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 >> 172.21.15.33:0/3833326189 conn(0x55d39ce529b0 msgr2=0x55d39ce54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 shutdown_connections 2024-01-25T16:09:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 1 -- 172.21.15.33:0/3833326189 wait complete. 2024-01-25T16:09:50.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.813+0000 7f7ac49f5740 20 client.15558 trim_cache size 0 max 16384 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.826+0000 7f7ac49f5740 1 Processor -- start 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 -- start start 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce58090 0x55d39ce60e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce599c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce4e270 con 0x55d39ce61780 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce6fdc0 con 0x55d39ce589b0 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce78d20 con 0x55d39ce58090 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44808/0 (socket says 172.21.15.33:44808) 2024-01-25T16:09:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 -- 172.21.15.33:0/2046142291 learned_addr learned my addr 172.21.15.33:0/2046142291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce58090 0x55d39ce60e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 -- 172.21.15.33:0/2046142291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce58090 msgr2=0x55d39ce60e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce58090 0x55d39ce60e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abaffd640 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 -- 172.21.15.33:0/2046142291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 msgr2=0x55d39ce599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 -- 172.21.15.33:0/2046142291 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d39cd5b060 con 0x55d39ce61780 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7abbfff640 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40cabe0 tx=0x7f7ab4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e31d9483cb3714 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7aa17fa640 1 -- 172.21.15.33:0/2046142291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab40a2570 con 0x55d39ce61780 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7aa17fa640 1 -- 172.21.15.33:0/2046142291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab400fd80 con 0x55d39ce61780 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.827+0000 7f7aa17fa640 1 -- 172.21.15.33:0/2046142291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab4013c20 con 0x55d39ce61780 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2046142291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 msgr2=0x55d39ce59f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7ab40cabe0 tx=0x7f7ab4007780 comp rx=0 tx=0).stop 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2046142291 shutdown_connections 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce61780 0x55d39ce59f00 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/2046142291 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce58090 0x55d39ce60e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2046142291 >> 172.21.15.33:0/2046142291 conn(0x55d39ce529b0 msgr2=0x55d39ce52d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2046142291 shutdown_connections 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- 172.21.15.33:0/2046142291 wait complete. 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.828+0000 7f7ac49f5740 1 -- start start 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 Processor -- start 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce68910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce66860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce66f30 con 0x55d39ce58090 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce670a0 con 0x55d39ce589b0 2024-01-25T16:09:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d39ce67260 con 0x55d39ce61780 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7ac49f5740 10 client.0 _ll_register_callbacks cb 0x7f7ab4095ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44824/0 (socket says 172.21.15.33:44824) 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 -- 172.21.15.33:0/1056315455 learned_addr learned my addr 172.21.15.33:0/1056315455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abbfff640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce66860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abaffd640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce68910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 -- 172.21.15.33:0/1056315455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 msgr2=0x55d39ce66860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce66860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 -- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 msgr2=0x55d39ce68910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce68910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 -- 172.21.15.33:0/1056315455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d39cdb7e70 con 0x55d39ce58090 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abbfff640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce66860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abaffd640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce68910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:09:50.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7abb7fe640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7aac001eb0 tx=0x7f7aac005b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5dabfb50158d7a4 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.829+0000 7f7a9affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac02b020 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7aac0050d0 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ab409e330 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7aac017040 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 10 client.15564 fetch_fsmap learned FSMap version 24 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55d39cd80a80 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ab409e330 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7aac003710 con 0x55d39ce58090 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 10 client.15564 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 10 client.15564 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 20 client.15564 populate_metadata read hostname 'smithi033' 2024-01-25T16:09:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39cec8fc0 0x55d39cecb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 4 client.15564 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d39cabf600 con 0x55d39cec8fc0 2024-01-25T16:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.830+0000 7f7abaffd640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39cec8fc0 0x55d39cecb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.831+0000 7f7a9affd640 10 client.15564 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:50.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.831+0000 7f7abaffd640 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39cec8fc0 0x55d39cecb3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62439874fe8f9ddc server_cookie=5bbbfda03721d697 in_seq=0 out_seq=0 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.859+0000 7f7a9affd640 1 -- 172.21.15.33:0/1056315455 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d39cabf600 con 0x55d39cec8fc0 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.859+0000 7f7a9affd640 10 client.15564 handle_command_reply: tid=0 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.859+0000 7f7ac49f5740 1 client.15564 shutdown 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 msgr2=0x55d39ce683d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7aac001eb0 tx=0x7f7aac005b60 comp rx=0 tx=0).stop 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 shutdown_connections 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d39cec8fc0 0x55d39cecb3e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d39ce61780 0x55d39ce66860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d39ce589b0 0x55d39ce68910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7a9affd640 0 client.15564 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 --2- 172.21.15.33:0/1056315455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d39ce58090 0x55d39ce683d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 >> 172.21.15.33:0/1056315455 conn(0x55d39ce529b0 msgr2=0x55d39ce599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 shutdown_connections 2024-01-25T16:09:50.863 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 1 -- 172.21.15.33:0/1056315455 wait complete. 2024-01-25T16:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac49f5740 20 client.15564 trim_cache size 0 max 16384 2024-01-25T16:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:09:50.864 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 msgr2=0x7f7aa40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 0x7f7aa40a39b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7abc076fd0 tx=0x7f7ab0004a30 comp rx=0 tx=0).stop 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 msgr2=0x7f7abc19b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7ab400cab0 tx=0x7f7ab400cf70 comp rx=0 tx=0).stop 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 shutdown_connections 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7aa40a14f0 0x7f7aa40a39b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7abc106de0 0x7f7abc19b960 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7abc068d10 0x7f7abc1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.865 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 --2- 172.21.15.33:0/2479703629 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7abc068370 0x7f7abc1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.860+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 >> 172.21.15.33:0/2479703629 conn(0x7f7abc076a40 msgr2=0x7f7abc0fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.861+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 shutdown_connections 2024-01-25T16:09:50.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:09:50.861+0000 7f7ac1e2b640 1 -- 172.21.15.33:0/2479703629 wait complete. 2024-01-25T16:09:51.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:51 smithi076 ceph-mon[35642]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 189 KiB/s wr, 46 op/s 2024-01-25T16:09:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:51 smithi033 ceph-mon[31417]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 189 KiB/s wr, 46 op/s 2024-01-25T16:09:51.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:51 smithi152 ceph-mon[36015]: pgmap v63: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 189 KiB/s wr, 46 op/s 2024-01-25T16:09:51.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23754 2.98 MB/sec warmup 48 sec latency 1283.515 ms 2024-01-25T16:09:52.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23754 2.92 MB/sec warmup 49 sec latency 2283.570 ms 2024-01-25T16:09:53.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:53 smithi076 ceph-mon[35642]: pgmap v64: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-01-25T16:09:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:53 smithi033 ceph-mon[31417]: pgmap v64: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-01-25T16:09:53.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:53 smithi152 ceph-mon[36015]: pgmap v64: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 209 KiB/s wr, 65 op/s 2024-01-25T16:09:53.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23754 2.86 MB/sec warmup 50 sec latency 3283.710 ms 2024-01-25T16:09:54.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 23754 2.80 MB/sec warmup 51 sec latency 4283.831 ms 2024-01-25T16:09:55.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:55 smithi076 ceph-mon[35642]: pgmap v65: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:55 smithi033 ceph-mon[31417]: pgmap v65: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:55 smithi152 ceph-mon[36015]: pgmap v65: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:55.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 24695 2.89 MB/sec warmup 52 sec latency 4956.651 ms 2024-01-25T16:09:56.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 27209 3.05 MB/sec warmup 53 sec latency 12.105 ms 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.419 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:57 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:09:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:57 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: pgmap v66: 97 pgs: 97 active+clean; 27 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 141 KiB/s wr, 36 op/s 2024-01-25T16:09:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:09:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:57 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:57.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 27210 2.99 MB/sec warmup 54 sec latency 999.901 ms 2024-01-25T16:09:58.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:58 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:09:58.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:58 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:09:58.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:58 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:58 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:09:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:58 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:09:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:58 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:58.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:58 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:09:58.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:58 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:09:58.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:58 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:09:58.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 27210 2.94 MB/sec warmup 55 sec latency 1999.999 ms 2024-01-25T16:09:59.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:09:59 smithi076 ceph-mon[35642]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 199 KiB/s wr, 50 op/s 2024-01-25T16:09:59.534 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:09:59 smithi152 ceph-mon[36015]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 199 KiB/s wr, 50 op/s 2024-01-25T16:09:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:09:59 smithi033 ceph-mon[31417]: pgmap v67: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 199 KiB/s wr, 50 op/s 2024-01-25T16:09:59.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 27210 2.89 MB/sec warmup 56 sec latency 3000.084 ms 2024-01-25T16:10:00.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:00 smithi033 ceph-mon[31417]: overall HEALTH_OK 2024-01-25T16:10:00.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:00 smithi152 ceph-mon[36015]: overall HEALTH_OK 2024-01-25T16:10:00.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:00 smithi076 ceph-mon[35642]: overall HEALTH_OK 2024-01-25T16:10:00.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 27304 2.84 MB/sec warmup 57 sec latency 3686.896 ms 2024-01-25T16:10:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:01 smithi033 ceph-mon[31417]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 78 KiB/s wr, 33 op/s 2024-01-25T16:10:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:01 smithi152 ceph-mon[36015]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 78 KiB/s wr, 33 op/s 2024-01-25T16:10:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:01 smithi076 ceph-mon[35642]: pgmap v68: 97 pgs: 97 active+clean; 27 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 78 KiB/s wr, 33 op/s 2024-01-25T16:10:01.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 29330 3.01 MB/sec warmup 58 sec latency 245.247 ms 2024-01-25T16:10:02.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 30807 3.07 MB/sec warmup 59 sec latency 619.475 ms 2024-01-25T16:10:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:03 smithi033 ceph-mon[31417]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 535 KiB/s wr, 78 op/s 2024-01-25T16:10:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:03 smithi152 ceph-mon[36015]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 535 KiB/s wr, 78 op/s 2024-01-25T16:10:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:03 smithi076 ceph-mon[35642]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 535 KiB/s wr, 78 op/s 2024-01-25T16:10:03.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 30807 3.02 MB/sec warmup 60 sec latency 1619.578 ms 2024-01-25T16:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 -- 172.21.15.33:0/3449051792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda140730a0 msgr2=0x7fda14073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 --2- 172.21.15.33:0/3449051792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda140730a0 0x7fda14073480 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fda04009900 tx=0x7fda0402bcd0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.963 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 -- 172.21.15.33:0/3449051792 shutdown_connections 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 --2- 172.21.15.33:0/3449051792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda1410d560 0x7fda1410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 --2- 172.21.15.33:0/3449051792 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140739c0 0x7fda1410d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 --2- 172.21.15.33:0/3449051792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda140730a0 0x7fda14073480 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 -- 172.21.15.33:0/3449051792 >> 172.21.15.33:0/3449051792 conn(0x7fda140fc7b0 msgr2=0x7fda140febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 -- 172.21.15.33:0/3449051792 shutdown_connections 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 -- 172.21.15.33:0/3449051792 wait complete. 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.958+0000 7fda197c4640 1 Processor -- start 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 -- start start 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140730a0 0x7fda14199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 0x7fda14199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.964 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419e2b0 con 0x7fda1410d560 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419e420 con 0x7fda140739c0 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda197c4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1419e5e0 con 0x7fda140730a0 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60576/0 (socket says 172.21.15.33:60576) 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda12ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140730a0 0x7fda14199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 -- 172.21.15.33:0/1109549285 learned_addr learned my addr 172.21.15.33:0/1109549285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda127fc640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 0x7fda14199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 -- 172.21.15.33:0/1109549285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140730a0 msgr2=0x7fda14199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140730a0 0x7fda14199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 -- 172.21.15.33:0/1109549285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 msgr2=0x7fda14199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.965 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 0x7fda14199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda137fe640 1 -- 172.21.15.33:0/1109549285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04009560 con 0x7fda1410d560 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.959+0000 7fda127fc640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 0x7fda14199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fda137fe640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00b970 tx=0x7fda0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fd9fbfff640 1 -- 172.21.15.33:0/1109549285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c00cb40 con 0x7fda1410d560 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fd9fbfff640 1 -- 172.21.15.33:0/1109549285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0c00cca0 con 0x7fda1410d560 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1419e8c0 con 0x7fda1410d560 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fd9fbfff640 1 -- 172.21.15.33:0/1109549285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c007770 con 0x7fda1410d560 2024-01-25T16:10:03.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.960+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141a6140 con 0x7fda1410d560 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.962+0000 7fd9fbfff640 1 -- 172.21.15.33:0/1109549285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fda0c014440 con 0x7fda1410d560 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 Processor -- start 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 -- start start 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fd9fbfff640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 0x7fd9fc0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b15240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b15780 0x560b15b17b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a7b070 con 0x560b15b0c090 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a78760 con 0x560b15b0c9b0 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda1c384740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a74050 con 0x560b15b15780 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda12ffd640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 0x7fd9fc0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60586/0 (socket says 172.21.15.33:60586) 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 -- 172.21.15.33:0/17026469 learned_addr learned my addr 172.21.15.33:0/17026469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda137fe640 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b15780 0x560b15b17b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b15240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fd9fbfff640 1 -- 172.21.15.33:0/1109549285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda0c0cc6d0 con 0x7fda1410d560 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 -- 172.21.15.33:0/17026469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b15780 msgr2=0x560b15b17b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b15780 0x560b15b17b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 -- 172.21.15.33:0/17026469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 msgr2=0x560b15b15240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b15240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.963+0000 7fda127fc640 1 -- 172.21.15.33:0/17026469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b15a6be70 con 0x560b15b0c090 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fda12ffd640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 0x7fd9fc0a3a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda0402c760 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fda127fc640 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fda080135a0 tx=0x7fda0800b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dfb9d4a6c6dcb1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fd9f97fa640 1 -- 172.21.15.33:0/17026469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08004030 con 0x560b15b0c090 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fd9f97fa640 1 -- 172.21.15.33:0/17026469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda080035f0 con 0x560b15b0c090 2024-01-25T16:10:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fd9f97fa640 1 -- 172.21.15.33:0/17026469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0800f650 con 0x560b15b0c090 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fda1c384740 1 -- 172.21.15.33:0/17026469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 msgr2=0x560b15b0c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fda1c384740 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fda080135a0 tx=0x7fda0800b490 comp rx=0 tx=0).stop 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.964+0000 7fda1c384740 1 -- 172.21.15.33:0/17026469 shutdown_connections 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b15780 0x560b15b17b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b15240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- 172.21.15.33:0/17026469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b0c470 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- 172.21.15.33:0/17026469 >> 172.21.15.33:0/17026469 conn(0x560b15b069b0 msgr2=0x560b15b06d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- 172.21.15.33:0/17026469 shutdown_connections 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- 172.21.15.33:0/17026469 wait complete. 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- start start 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 Processor -- start 2024-01-25T16:10:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b13110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a2d1e0 con 0x560b15b0c9b0 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a2ce80 con 0x560b15b15780 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b159f20c0 con 0x560b15b0c090 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda1c384740 10 client.0 _ll_register_callbacks cb 0x560b15a17ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60594/0 (socket says 172.21.15.33:60594) 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda127fc640 1 -- 172.21.15.33:0/82754412 learned_addr learned my addr 172.21.15.33:0/82754412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.965+0000 7fda137fe640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b13110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda12ffd640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 -- 172.21.15.33:0/82754412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 msgr2=0x560b15b12690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b12690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 -- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 msgr2=0x560b15b13110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b13110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 -- 172.21.15.33:0/82754412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0800be50 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda12ffd640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda127fc640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda08004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1996c92b8ec137 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08004510 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda08003970 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b15a6d040 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda08003b30 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda1c384740 10 client.15576 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560b15a726d0 con 0x560b15b0c9b0 2024-01-25T16:10:03.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560b15a6d040 con 0x560b15b0c9b0 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.966+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda0802f070 con 0x560b15b0c9b0 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 10 client.15576 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 20 client.15576 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b25420 0x560b15b27840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 4 client.15576 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b15a6cd60 con 0x560b15b25420 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda12ffd640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b25420 0x560b15b27840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fda12ffd640 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b25420 0x560b15b27840 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa5b149359de2912 server_cookie=d53d4d9702701dde in_seq=0 out_seq=0 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.967+0000 7fd9f6ff5640 10 client.15576 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/82754412 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560b15a6cd60 con 0x560b15b25420 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fd9f6ff5640 10 client.15576 handle_command_reply: tid=0 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 client.15576 shutdown 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 msgr2=0x560b15b12bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fda08006010 tx=0x7fda08004320 comp rx=0 tx=0).stop 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 shutdown_connections 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b25420 0x560b15b27840 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b13110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c9b0 0x560b15b12bd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fd9f6ff5640 0 client.15576 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 --2- 172.21.15.33:0/82754412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 >> 172.21.15.33:0/82754412 conn(0x560b15b069b0 msgr2=0x560b15b08700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 shutdown_connections 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 1 -- 172.21.15.33:0/82754412 wait complete. 2024-01-25T16:10:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.968+0000 7fda1c384740 20 client.15576 trim_cache size 0 max 16384 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 Processor -- start 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 -- start start 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b12f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15a6d040 con 0x560b15b15780 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15b0b370 con 0x560b15b0c9b0 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda1c384740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15b00e60 con 0x560b15b0c090 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60608/0 (socket says 172.21.15.33:60608) 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 -- 172.21.15.33:0/1307148808 learned_addr learned my addr 172.21.15.33:0/1307148808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda12ffd640 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda127fc640 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b12f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 -- 172.21.15.33:0/1307148808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 msgr2=0x560b15b117f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b117f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 -- 172.21.15.33:0/1307148808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 msgr2=0x560b15b12f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b12f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.982+0000 7fda137fe640 1 -- 172.21.15.33:0/1307148808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b15a0f060 con 0x560b15b15780 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda137fe640 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00b940 tx=0x7fda0c00e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112336f8f94fa235 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fd9f8ff9640 1 -- 172.21.15.33:0/1307148808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0840b0 con 0x560b15b15780 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fd9f8ff9640 1 -- 172.21.15.33:0/1307148808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda0c099640 con 0x560b15b15780 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fd9f8ff9640 1 -- 172.21.15.33:0/1307148808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0c0a19b0 con 0x560b15b15780 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 -- 172.21.15.33:0/1307148808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 msgr2=0x560b15b1c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00b940 tx=0x7fda0c00e040 comp rx=0 tx=0).stop 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 -- 172.21.15.33:0/1307148808 shutdown_connections 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b15780 0x560b15b1c410 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b0c9b0 0x560b15b12f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 --2- 172.21.15.33:0/1307148808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c090 0x560b15b117f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 -- 172.21.15.33:0/1307148808 >> 172.21.15.33:0/1307148808 conn(0x560b15b069b0 msgr2=0x560b15b051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 -- 172.21.15.33:0/1307148808 shutdown_connections 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.983+0000 7fda1c384740 1 -- 172.21.15.33:0/1307148808 wait complete. 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 -- start start 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 Processor -- start 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c9b0 0x560b15b1bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b7b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15b1c320 con 0x560b15b0c090 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15b7bee0 con 0x560b15b15780 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b15b7c0a0 con 0x560b15b0c9b0 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda1c384740 10 client.0 _ll_register_callbacks cb 0x7fda0800b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60612/0 (socket says 172.21.15.33:60612) 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 -- 172.21.15.33:0/865058908 learned_addr learned my addr 172.21.15.33:0/865058908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda127fc640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c9b0 0x560b15b1bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda137fe640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b7b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 -- 172.21.15.33:0/865058908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c9b0 msgr2=0x560b15b1bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c9b0 0x560b15b1bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 -- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 msgr2=0x560b15b7b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b7b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda12ffd640 1 -- 172.21.15.33:0/865058908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda141a6140 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.984+0000 7fda137fe640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b7b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda12ffd640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda04009230 tx=0x7fda0402bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98b2549165091bc server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda04046040 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fda040044a0 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda080035f0 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda0402ca30 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 10 client.15582 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560b15a34a80 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda080035f0 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fda0404f280 con 0x560b15b0c090 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 10 client.15582 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 20 client.15582 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b7d8e0 0x560b15b7fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 4 client.15582 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.985+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560b15773600 con 0x560b15b7d8e0 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.986+0000 7fda127fc640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b7d8e0 0x560b15b7fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.986+0000 7fda127fc640 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b7d8e0 0x560b15b7fd00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ddd2eedd0f4c1ad server_cookie=8cbf9106b6989b4b in_seq=0 out_seq=0 2024-01-25T16:10:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:03.986+0000 7fd9f6ff5640 10 client.15582 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fd9f6ff5640 1 -- 172.21.15.33:0/865058908 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560b15773600 con 0x560b15b7d8e0 2024-01-25T16:10:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fd9f6ff5640 10 client.15582 handle_command_reply: tid=0 2024-01-25T16:10:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 client.15582 shutdown 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 msgr2=0x560b15b1b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fda04009230 tx=0x7fda0402bcd0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 shutdown_connections 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560b15b7d8e0 0x560b15b7fd00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560b15b15780 0x560b15b7b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560b15b0c9b0 0x560b15b1bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 --2- 172.21.15.33:0/865058908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560b15b0c090 0x560b15b1b650 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fd9f6ff5640 0 client.15582 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 >> 172.21.15.33:0/865058908 conn(0x560b15b069b0 msgr2=0x560b15b05a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.036+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 shutdown_connections 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda1c384740 1 -- 172.21.15.33:0/865058908 wait complete. 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda1c384740 20 client.15582 trim_cache size 0 max 16384 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:10:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 msgr2=0x7fd9fc0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 0x7fd9fc0a3a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fda04006010 tx=0x7fda0402c760 comp rx=0 tx=0).stop 2024-01-25T16:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 msgr2=0x7fda1419dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00b970 tx=0x7fda0c00be30 comp rx=0 tx=0).stop 2024-01-25T16:10:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 shutdown_connections 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd9fc0a1570 0x7fd9fc0a3a30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda1410d560 0x7fda1419dcb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda140739c0 0x7fda14199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 --2- 172.21.15.33:0/1109549285 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fda140730a0 0x7fda14199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 >> 172.21.15.33:0/1109549285 conn(0x7fda140fc7b0 msgr2=0x7fda140fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 shutdown_connections 2024-01-25T16:10:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:04.037+0000 7fda197c4640 1 -- 172.21.15.33:0/1109549285 wait complete. 2024-01-25T16:10:04.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 30807 2.97 MB/sec warmup 61 sec latency 2619.688 ms 2024-01-25T16:10:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:05 smithi033 ceph-mon[31417]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:05.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:05 smithi152 ceph-mon[36015]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:05 smithi076 ceph-mon[35642]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:05.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 31430 3.01 MB/sec warmup 62 sec latency 3310.847 ms 2024-01-25T16:10:06.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 33573 3.15 MB/sec warmup 63 sec latency 13.396 ms 2024-01-25T16:10:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:07 smithi033 ceph-mon[31417]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:07 smithi152 ceph-mon[36015]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:07 smithi076 ceph-mon[35642]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 489 MiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 515 KiB/s wr, 58 op/s 2024-01-25T16:10:07.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34300 3.12 MB/sec warmup 64 sec latency 862.763 ms 2024-01-25T16:10:08.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34300 3.08 MB/sec warmup 65 sec latency 1862.864 ms 2024-01-25T16:10:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:09 smithi033 ceph-mon[31417]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 KiB/s wr, 133 op/s 2024-01-25T16:10:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:09 smithi152 ceph-mon[36015]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 KiB/s wr, 133 op/s 2024-01-25T16:10:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:09 smithi076 ceph-mon[35642]: pgmap v72: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 KiB/s wr, 133 op/s 2024-01-25T16:10:09.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34300 3.03 MB/sec warmup 66 sec latency 2862.952 ms 2024-01-25T16:10:10.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34455 3.00 MB/sec warmup 67 sec latency 3545.837 ms 2024-01-25T16:10:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:11 smithi033 ceph-mon[31417]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 118 op/s 2024-01-25T16:10:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:11 smithi152 ceph-mon[36015]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 118 op/s 2024-01-25T16:10:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:11 smithi076 ceph-mon[35642]: pgmap v73: 97 pgs: 97 active+clean; 28 MiB data, 478 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 786 KiB/s wr, 118 op/s 2024-01-25T16:10:11.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34455 2.95 MB/sec warmup 68 sec latency 1172.506 ms 2024-01-25T16:10:12.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34455 2.91 MB/sec warmup 69 sec latency 2172.591 ms 2024-01-25T16:10:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:13 smithi033 ceph-mon[31417]: pgmap v74: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 957 KiB/s wr, 146 op/s 2024-01-25T16:10:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:13 smithi152 ceph-mon[36015]: pgmap v74: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 957 KiB/s wr, 146 op/s 2024-01-25T16:10:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:13 smithi076 ceph-mon[35642]: pgmap v74: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 957 KiB/s wr, 146 op/s 2024-01-25T16:10:13.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34455 2.87 MB/sec warmup 70 sec latency 3172.690 ms 2024-01-25T16:10:14.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:14 smithi033 ceph-mon[31417]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:14.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:14 smithi152 ceph-mon[36015]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:14.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:14 smithi076 ceph-mon[35642]: pgmap v75: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:14.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 34455 2.83 MB/sec warmup 71 sec latency 4172.738 ms 2024-01-25T16:10:15.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 35506 2.92 MB/sec warmup 72 sec latency 4846.617 ms 2024-01-25T16:10:16.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 37903 3.00 MB/sec warmup 73 sec latency 16.176 ms 2024-01-25T16:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 -- 172.21.15.33:0/3587199766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 msgr2=0x7ffb3c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 --2- 172.21.15.33:0/3587199766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c0687d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009900 tx=0x7ffb3002bc30 comp rx=0 tx=0).stop 2024-01-25T16:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 -- 172.21.15.33:0/3587199766 shutdown_connections 2024-01-25T16:10:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 --2- 172.21.15.33:0/3587199766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 0x7ffb3c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 --2- 172.21.15.33:0/3587199766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c0687d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 --2- 172.21.15.33:0/3587199766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 0x7ffb3c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 -- 172.21.15.33:0/3587199766 >> 172.21.15.33:0/3587199766 conn(0x7ffb3c076a40 msgr2=0x7ffb3c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 -- 172.21.15.33:0/3587199766 shutdown_connections 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.124+0000 7ffb42905640 1 -- 172.21.15.33:0/3587199766 wait complete. 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 Processor -- start 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 -- start start 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 0x7ffb3c197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 0x7ffb3c19ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c19c010 con 0x7ffb3c068370 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c19c180 con 0x7ffb3c106e00 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb42905640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c19c340 con 0x7ffb3c068d10 2024-01-25T16:10:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58750/0 (socket says 172.21.15.33:58750) 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb3bfff640 1 -- 172.21.15.33:0/977464036 learned_addr learned my addr 172.21.15.33:0/977464036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.125+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 0x7ffb3c197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 0x7ffb3c19ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 -- 172.21.15.33:0/977464036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 msgr2=0x7ffb3c197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 0x7ffb3c197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 -- 172.21.15.33:0/977464036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 msgr2=0x7ffb3c19ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 0x7ffb3c19ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 -- 172.21.15.33:0/977464036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30009560 con 0x7ffb3c068370 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00e9f0 tx=0x7ffb2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb397fa640 1 -- 172.21.15.33:0/977464036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c009700 con 0x7ffb3c068370 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c19c620 con 0x7ffb3c068370 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb397fa640 1 -- 172.21.15.33:0/977464036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb2c00cd80 con 0x7ffb3c068370 2024-01-25T16:10:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb397fa640 1 -- 172.21.15.33:0/977464036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c018860 con 0x7ffb3c068370 2024-01-25T16:10:17.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.126+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c1a3ea0 con 0x7ffb3c068370 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.128+0000 7ffb397fa640 1 -- 172.21.15.33:0/977464036 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ffb2c0189c0 con 0x7ffb3c068370 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 Processor -- start 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 -- start start 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb397fa640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 0x7ffb240a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 0x557ae6a08470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 0x557ae6a11240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a11780 0x557ae6a13b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6977070 con 0x557ae6a08090 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6974760 con 0x557ae6a089b0 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb455a1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6970050 con 0x557ae6a11780 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 0x557ae6a08470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb397fa640 1 -- 172.21.15.33:0/977464036 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffb2c014070 con 0x7ffb3c068370 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 0x7ffb240a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 0x557ae6a11240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 0x557ae6a08470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58756/0 (socket says 172.21.15.33:58756) 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3bfff640 1 -- 172.21.15.33:0/2662283460 learned_addr learned my addr 172.21.15.33:0/2662283460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a11780 0x557ae6a13b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/2662283460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a11780 msgr2=0x557ae6a13b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a11780 0x557ae6a13b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/2662283460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 msgr2=0x557ae6a08470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 0x557ae6a08470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.129+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/2662283460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae6967e70 con 0x557ae6a089b0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.130+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 0x7ffb240a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x557ae6a03dd0 tx=0x7ffb34009790 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.130+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 0x557ae6a11240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000c00 tx=0x7ffb30004dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5401b80d29f57c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.130+0000 7ffb227fc640 1 -- 172.21.15.33:0/2662283460 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004740 con 0x557ae6a089b0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.130+0000 7ffb227fc640 1 -- 172.21.15.33:0/2662283460 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb300048a0 con 0x557ae6a089b0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.130+0000 7ffb227fc640 1 -- 172.21.15.33:0/2662283460 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30005200 con 0x557ae6a089b0 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- 172.21.15.33:0/2662283460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 msgr2=0x557ae6a11240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 0x557ae6a11240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000c00 tx=0x7ffb30004dd0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- 172.21.15.33:0/2662283460 shutdown_connections 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a11780 0x557ae6a13b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a089b0 0x557ae6a11240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 --2- 172.21.15.33:0/2662283460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a08090 0x557ae6a08470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- 172.21.15.33:0/2662283460 >> 172.21.15.33:0/2662283460 conn(0x557ae6a029b0 msgr2=0x557ae6a02d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- 172.21.15.33:0/2662283460 shutdown_connections 2024-01-25T16:10:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- 172.21.15.33:0/2662283460 wait complete. 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.131+0000 7ffb455a1740 1 -- start start 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 Processor -- start 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a089b0 0x557ae6a0eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae69291e0 con 0x557ae6a11780 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6928e80 con 0x557ae6a08090 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae68ee0c0 con 0x557ae6a089b0 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb455a1740 10 client.0 _ll_register_callbacks cb 0x557ae6913ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb40ea2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb40ea2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58758/0 (socket says 172.21.15.33:58758) 2024-01-25T16:10:17.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:48982/0 (socket says 172.21.15.33:48982) 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb40ea2640 1 -- 172.21.15.33:0/1856797997 learned_addr learned my addr 172.21.15.33:0/1856797997 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a089b0 0x557ae6a0eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 -- 172.21.15.33:0/1856797997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a089b0 msgr2=0x557ae6a0eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a089b0 0x557ae6a0eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 -- 172.21.15.33:0/1856797997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 msgr2=0x557ae6a0f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb3bfff640 1 -- 172.21.15.33:0/1856797997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c10e480 con 0x557ae6a08090 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.132+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0caf80 tx=0x7ffb2c09a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b342345ab2078cc server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0cac00 con 0x557ae6a08090 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ae6969040 con 0x557ae6a08090 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffb2c0cad60 con 0x557ae6a08090 2024-01-25T16:10:17.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb2c0967f0 con 0x557ae6a08090 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 10 client.25151 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ae696e6d0 con 0x557ae6a08090 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ae6969040 con 0x557ae6a08090 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb2c0a1f30 con 0x557ae6a08090 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 10 client.25151 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 10 client.25151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 20 client.25151 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a21450 0x557ae6a23840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 4 client.25151 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ae6968d60 con 0x557ae6a21450 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.133+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a21450 0x557ae6a23840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.134+0000 7ffb1b7fe640 10 client.25151 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.134+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a21450 0x557ae6a23840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61e7e7f9f8f8d89 server_cookie=5c544e30d4631278 in_seq=0 out_seq=0 2024-01-25T16:10:17.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.134+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/1856797997 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ae6968d60 con 0x557ae6a21450 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.134+0000 7ffb1b7fe640 10 client.25151 handle_command_reply: tid=0 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.134+0000 7ffb455a1740 1 client.25151 shutdown 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 msgr2=0x557ae6a0e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0caf80 tx=0x7ffb2c09a0e0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 shutdown_connections 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a21450 0x557ae6a23840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a11780 0x557ae6a0f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a089b0 0x557ae6a0eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 --2- 172.21.15.33:0/1856797997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a08090 0x557ae6a0e660 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb1b7fe640 0 client.25151 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 >> 172.21.15.33:0/1856797997 conn(0x557ae6a029b0 msgr2=0x557ae6a04700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 shutdown_connections 2024-01-25T16:10:17.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 1 -- 172.21.15.33:0/1856797997 wait complete. 2024-01-25T16:10:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.135+0000 7ffb455a1740 20 client.25151 trim_cache size 0 max 16384 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 Processor -- start 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 -- start start 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a0b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6969040 con 0x557ae6a089b0 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6a21250 con 0x557ae6a11780 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb455a1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6a027e0 con 0x557ae6a08090 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58772/0 (socket says 172.21.15.33:58772) 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/3237248657 learned_addr learned my addr 172.21.15.33:0/3237248657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a0b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/3237248657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 msgr2=0x557ae6a10d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a10d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/3237248657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 msgr2=0x557ae6a0b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a0b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/3237248657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ae690b060 con 0x557ae6a089b0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a10d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.148+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009230 tx=0x7ffb30041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ba41612b4f3875 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb21ffb640 1 -- 172.21.15.33:0/3237248657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3002c980 con 0x557ae6a089b0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb21ffb640 1 -- 172.21.15.33:0/3237248657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb30004500 con 0x557ae6a089b0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb21ffb640 1 -- 172.21.15.33:0/3237248657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30004ec0 con 0x557ae6a089b0 2024-01-25T16:10:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- 172.21.15.33:0/3237248657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 msgr2=0x557ae6a0ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009230 tx=0x7ffb30041040 comp rx=0 tx=0).stop 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- 172.21.15.33:0/3237248657 shutdown_connections 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a0b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a0ef00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 --2- 172.21.15.33:0/3237248657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a10d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- 172.21.15.33:0/3237248657 >> 172.21.15.33:0/3237248657 conn(0x557ae6a029b0 msgr2=0x557ae6a09980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- 172.21.15.33:0/3237248657 shutdown_connections 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- 172.21.15.33:0/3237248657 wait complete. 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.149+0000 7ffb455a1740 1 -- start start 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 Processor -- start 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a160f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a16b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6a769e0 con 0x557ae6a089b0 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6a76b50 con 0x557ae6a11780 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ae6a76d10 con 0x557ae6a08090 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb455a1740 10 client.0 _ll_register_callbacks cb 0x7ffb30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58784/0 (socket says 172.21.15.33:58784) 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/4170134908 learned_addr learned my addr 172.21.15.33:0/4170134908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb40ea2640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a16b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/4170134908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 msgr2=0x557ae6a160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 msgr2=0x557ae6a16b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a16b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.150+0000 7ffb3b7fe640 1 -- 172.21.15.33:0/4170134908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ae6967e70 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb3b7fe640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000c00 tx=0x7ffb30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0a226d488cafb4 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30060070 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7ffb3004f040 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ae666f600 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb30053900 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 10 client.15600 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557ae6930a80 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ae666f600 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ffb30063020 con 0x557ae6a089b0 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 10 client.15600 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 10 client.15600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 20 client.15600 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a78030 0x557ae6a7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 4 client.15600 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ae66c2c50 con 0x557ae6a78030 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.151+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a78030 0x557ae6a7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.152+0000 7ffb3bfff640 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a78030 0x557ae6a7a420 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f005a242458597 server_cookie=b4394511e6e8f342 in_seq=0 out_seq=0 2024-01-25T16:10:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.152+0000 7ffb1b7fe640 10 client.15600 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb1b7fe640 1 -- 172.21.15.33:0/4170134908 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ae66c2c50 con 0x557ae6a78030 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb1b7fe640 10 client.15600 handle_command_reply: tid=0 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 client.15600 shutdown 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 msgr2=0x557ae6a16630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffb30000c00 tx=0x7ffb30009260 comp rx=0 tx=0).stop 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 shutdown_connections 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557ae6a78030 0x557ae6a7a420 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae6a11780 0x557ae6a16b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ae6a089b0 0x557ae6a16630 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb1b7fe640 0 client.15600 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:17.202 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 --2- 172.21.15.33:0/4170134908 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557ae6a08090 0x557ae6a160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 >> 172.21.15.33:0/4170134908 conn(0x557ae6a029b0 msgr2=0x557ae6a0ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 shutdown_connections 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 1 -- 172.21.15.33:0/4170134908 wait complete. 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.199+0000 7ffb455a1740 20 client.15600 trim_cache size 0 max 16384 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:10:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:10:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 msgr2=0x7ffb240a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 0x7ffb240a39b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x557ae6a03dd0 tx=0x7ffb34009790 comp rx=0 tx=0).stop 2024-01-25T16:10:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 msgr2=0x7ffb3c197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00e9f0 tx=0x7ffb2c00eeb0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 shutdown_connections 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffb240a14f0 0x7ffb240a39b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb3c106e00 0x7ffb3c19ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffb3c068d10 0x7ffb3c197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 --2- 172.21.15.33:0/977464036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb3c068370 0x7ffb3c197150 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 >> 172.21.15.33:0/977464036 conn(0x7ffb3c076a40 msgr2=0x7ffb3c0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 shutdown_connections 2024-01-25T16:10:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:17.200+0000 7ffb42905640 1 -- 172.21.15.33:0/977464036 wait complete. 2024-01-25T16:10:17.382 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:17 smithi152 ceph-mon[36015]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:17.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:17 smithi076 ceph-mon[35642]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:17 smithi033 ceph-mon[31417]: pgmap v76: 97 pgs: 97 active+clean; 29 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 500 KiB/s wr, 101 op/s 2024-01-25T16:10:17.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 37998 2.97 MB/sec warmup 74 sec latency 911.976 ms 2024-01-25T16:10:18.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 37998 2.93 MB/sec warmup 75 sec latency 1912.058 ms 2024-01-25T16:10:19.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:19 smithi076 ceph-mon[35642]: pgmap v77: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 KiB/s wr, 121 op/s 2024-01-25T16:10:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:19 smithi033 ceph-mon[31417]: pgmap v77: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 KiB/s wr, 121 op/s 2024-01-25T16:10:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:19 smithi152 ceph-mon[36015]: pgmap v77: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 754 KiB/s wr, 121 op/s 2024-01-25T16:10:19.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 37998 2.89 MB/sec warmup 76 sec latency 2912.153 ms 2024-01-25T16:10:20.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 39049 2.98 MB/sec warmup 77 sec latency 3583.507 ms 2024-01-25T16:10:21.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:21 smithi076 ceph-mon[35642]: pgmap v78: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 426 KiB/s wr, 46 op/s 2024-01-25T16:10:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:21 smithi033 ceph-mon[31417]: pgmap v78: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 426 KiB/s wr, 46 op/s 2024-01-25T16:10:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:21 smithi152 ceph-mon[36015]: pgmap v78: 97 pgs: 97 active+clean; 30 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 426 KiB/s wr, 46 op/s 2024-01-25T16:10:21.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 41459 3.06 MB/sec warmup 78 sec latency 13.725 ms 2024-01-25T16:10:22.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 43531 3.18 MB/sec warmup 79 sec latency 9.169 ms 2024-01-25T16:10:23.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:23 smithi076 ceph-mon[35642]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 105 op/s 2024-01-25T16:10:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:23 smithi033 ceph-mon[31417]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 105 op/s 2024-01-25T16:10:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:23 smithi152 ceph-mon[36015]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 660 KiB/s wr, 105 op/s 2024-01-25T16:10:23.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44933 3.22 MB/sec warmup 80 sec latency 658.508 ms 2024-01-25T16:10:24.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44933 3.18 MB/sec warmup 81 sec latency 1658.541 ms 2024-01-25T16:10:25.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:25 smithi076 ceph-mon[35642]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:25 smithi033 ceph-mon[31417]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:25 smithi152 ceph-mon[36015]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:25.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44987 3.14 MB/sec warmup 82 sec latency 2327.693 ms 2024-01-25T16:10:26.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44987 3.10 MB/sec warmup 83 sec latency 1291.864 ms 2024-01-25T16:10:27.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:27 smithi076 ceph-mon[35642]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:27 smithi033 ceph-mon[31417]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:27 smithi152 ceph-mon[36015]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 516 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 489 KiB/s wr, 78 op/s 2024-01-25T16:10:27.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44987 3.06 MB/sec warmup 84 sec latency 2291.953 ms 2024-01-25T16:10:28.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44987 3.03 MB/sec warmup 85 sec latency 3292.044 ms 2024-01-25T16:10:29.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:29 smithi076 ceph-mon[35642]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 119 op/s 2024-01-25T16:10:29.435 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:29 smithi152 ceph-mon[36015]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 119 op/s 2024-01-25T16:10:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:29 smithi033 ceph-mon[31417]: pgmap v82: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 792 KiB/s wr, 119 op/s 2024-01-25T16:10:29.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44987 2.99 MB/sec warmup 86 sec latency 4292.167 ms 2024-01-25T16:10:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 -- 172.21.15.33:0/2418714924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548100800 msgr2=0x7fc54810ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 --2- 172.21.15.33:0/2418714924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548100800 0x7fc54810ca20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc540009ca0 tx=0x7fc54002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 -- 172.21.15.33:0/2418714924 shutdown_connections 2024-01-25T16:10:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 --2- 172.21.15.33:0/2418714924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548100800 0x7fc54810ca20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.301 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 --2- 172.21.15.33:0/2418714924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 0x7fc5481002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 --2- 172.21.15.33:0/2418714924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548104b90 0x7fc548104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.295+0000 7fc54ec87640 1 -- 172.21.15.33:0/2418714924 >> 172.21.15.33:0/2418714924 conn(0x7fc548076a40 msgr2=0x7fc548076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- 172.21.15.33:0/2418714924 shutdown_connections 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- 172.21.15.33:0/2418714924 wait complete. 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 Processor -- start 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- start start 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 0x7fc5481972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548100800 0x7fc548197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819c180 con 0x7fc548104b90 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819c2f0 con 0x7fc548100800 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54ec87640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54819c4b0 con 0x7fc5480ffe40 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.296+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.302 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39286/0 (socket says 172.21.15.33:39286) 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 -- 172.21.15.33:0/4108830664 learned_addr learned my addr 172.21.15.33:0/4108830664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 0x7fc5481972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc547fff640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548100800 0x7fc548197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 -- 172.21.15.33:0/4108830664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 msgr2=0x7fc5481972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 0x7fc5481972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 -- 172.21.15.33:0/4108830664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548100800 msgr2=0x7fc548197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548100800 0x7fc548197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 -- 172.21.15.33:0/4108830664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc540009900 con 0x7fc548104b90 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54d224640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc54002c700 tx=0x7fc540035d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc545ffb640 1 -- 172.21.15.33:0/4108830664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540032a10 con 0x7fc548104b90 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54819c730 con 0x7fc548104b90 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc545ffb640 1 -- 172.21.15.33:0/4108830664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc540003e30 con 0x7fc548104b90 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.297+0000 7fc545ffb640 1 -- 172.21.15.33:0/4108830664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54003f440 con 0x7fc548104b90 2024-01-25T16:10:30.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.298+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5481a4010 con 0x7fc548104b90 2024-01-25T16:10:30.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.300+0000 7fc545ffb640 1 -- 172.21.15.33:0/4108830664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fc54003b040 con 0x7fc548104b90 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.300+0000 7fc551846740 1 Processor -- start 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 -- start start 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc545ffb640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 0x7fc5300a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f860a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f8613240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f8579070 con 0x5650f8613780 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f8576760 con 0x5650f860a9b0 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc551846740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f8572050 con 0x5650f860a090 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc547fff640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 0x7fc5300a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54ca23640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f860a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39298/0 (socket says 172.21.15.33:39298) 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 -- 172.21.15.33:0/3582945312 learned_addr learned my addr 172.21.15.33:0/3582945312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc547fff640 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f8613240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc545ffb640 1 -- 172.21.15.33:0/4108830664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc5400efb30 con 0x7fc548104b90 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 -- 172.21.15.33:0/3582945312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 msgr2=0x5650f860a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f860a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 -- 172.21.15.33:0/3582945312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 msgr2=0x5650f8613240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f8613240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.301+0000 7fc54d224640 1 -- 172.21.15.33:0/3582945312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f8569e70 con 0x5650f8613780 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc547fff640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 0x7fc5300a3a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc53c005fd0 tx=0x7fc53c00a710 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc54d224640 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc5400b9460 tx=0x7fc540004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2e220034bd2aca server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc52effd640 1 -- 172.21.15.33:0/3582945312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400bc530 con 0x5650f8613780 2024-01-25T16:10:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc52effd640 1 -- 172.21.15.33:0/3582945312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5400bc350 con 0x5650f8613780 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc52effd640 1 -- 172.21.15.33:0/3582945312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400c4d80 con 0x5650f8613780 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc551846740 1 -- 172.21.15.33:0/3582945312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 msgr2=0x5650f8615b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.302+0000 7fc551846740 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc5400b9460 tx=0x7fc540004040 comp rx=0 tx=0).stop 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- 172.21.15.33:0/3582945312 shutdown_connections 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8615b70 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f8613240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- 172.21.15.33:0/3582945312 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f860a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- 172.21.15.33:0/3582945312 >> 172.21.15.33:0/3582945312 conn(0x5650f86049b0 msgr2=0x5650f8604d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- 172.21.15.33:0/3582945312 shutdown_connections 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- 172.21.15.33:0/3582945312 wait complete. 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- start start 2024-01-25T16:10:30.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 Processor -- start 2024-01-25T16:10:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f8610650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 0x5650f86110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f852b1e0 con 0x5650f860a9b0 2024-01-25T16:10:30.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f852ae80 con 0x5650f8613780 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.303+0000 7fc551846740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f84f00c0 con 0x5650f860a090 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc551846740 10 client.0 _ll_register_callbacks cb 0x5650f8515ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39304/0 (socket says 172.21.15.33:39304) 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 -- 172.21.15.33:0/1174381538 learned_addr learned my addr 172.21.15.33:0/1174381538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc54d224640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 0x5650f86110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc54ca23640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f8610650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 -- 172.21.15.33:0/1174381538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 msgr2=0x5650f8610650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f8610650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 -- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 msgr2=0x5650f86110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 0x5650f86110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 -- 172.21.15.33:0/1174381538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5481a4010 con 0x5650f860a9b0 2024-01-25T16:10:30.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc54d224640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 0x5650f86110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc547fff640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc53c01c600 tx=0x7fc53c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922f4015ef7629da server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc527fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c005220 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc53c012e10 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650f856b040 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc53c0124f0 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc551846740 10 client.15612 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.304+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5650f85706d0 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650f856b040 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc53c0206a0 con 0x5650f860a9b0 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 10 client.15612 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 20 client.15612 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f86232a0 0x5650f86256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 4 client.15612 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650f856ad60 con 0x5650f86232a0 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.305+0000 7fc54ca23640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f86232a0 0x5650f86256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc527fff640 10 client.15612 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc54ca23640 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f86232a0 0x5650f86256c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b543f986310cdd server_cookie=a9dde8b1a83ad33d in_seq=0 out_seq=0 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc527fff640 1 -- 172.21.15.33:0/1174381538 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650f856ad60 con 0x5650f86232a0 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc527fff640 10 client.15612 handle_command_reply: tid=0 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc551846740 1 client.15612 shutdown 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.306+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 msgr2=0x5650f8610b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc53c01c600 tx=0x7fc53c01a000 comp rx=0 tx=0).stop 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 shutdown_connections 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f86232a0 0x5650f86256c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f8613780 0x5650f86110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a9b0 0x5650f8610b90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 --2- 172.21.15.33:0/1174381538 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a090 0x5650f8610650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc527fff640 0 client.15612 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 >> 172.21.15.33:0/1174381538 conn(0x5650f86049b0 msgr2=0x5650f8606700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 shutdown_connections 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 1 -- 172.21.15.33:0/1174381538 wait complete. 2024-01-25T16:10:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.307+0000 7fc551846740 20 client.15612 trim_cache size 0 max 16384 2024-01-25T16:10:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 Processor -- start 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 -- start start 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f861a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f8613780 0x5650f8610ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f856b040 con 0x5650f860a090 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f861ab50 con 0x5650f860a9b0 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.320+0000 7fc551846740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f8613600 con 0x5650f8613780 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39320/0 (socket says 172.21.15.33:39320) 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 -- 172.21.15.33:0/4056978901 learned_addr learned my addr 172.21.15.33:0/4056978901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 -- 172.21.15.33:0/4056978901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f8613780 msgr2=0x5650f8610ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f8613780 0x5650f8610ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 -- 172.21.15.33:0/4056978901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 msgr2=0x5650f861a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f861a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 -- 172.21.15.33:0/4056978901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650f850d060 con 0x5650f860a090 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc538006e10 tx=0x7fc538008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e064acc06183e7ce server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc52e7fc640 1 -- 172.21.15.33:0/4056978901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53801b070 con 0x5650f860a090 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc52e7fc640 1 -- 172.21.15.33:0/4056978901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc53801b810 con 0x5650f860a090 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.321+0000 7fc52e7fc640 1 -- 172.21.15.33:0/4056978901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc538016e60 con 0x5650f860a090 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- 172.21.15.33:0/4056978901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 msgr2=0x5650f8612d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc538006e10 tx=0x7fc538008040 comp rx=0 tx=0).stop 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- 172.21.15.33:0/4056978901 shutdown_connections 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f8613780 0x5650f8610ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a9b0 0x5650f861a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 --2- 172.21.15.33:0/4056978901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f860a090 0x5650f8612d80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- 172.21.15.33:0/4056978901 >> 172.21.15.33:0/4056978901 conn(0x5650f86049b0 msgr2=0x5650f860ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- 172.21.15.33:0/4056978901 shutdown_connections 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- 172.21.15.33:0/4056978901 wait complete. 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 -- start start 2024-01-25T16:10:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.322+0000 7fc551846740 1 Processor -- start 2024-01-25T16:10:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 0x5650f86778a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a9b0 0x5650f8677de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f86722c0 con 0x5650f8613780 2024-01-25T16:10:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f8672430 con 0x5650f860a090 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650f86725f0 con 0x5650f860a9b0 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 10 client.0 _ll_register_callbacks cb 0x7fc540005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39332/0 (socket says 172.21.15.33:39332) 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 -- 172.21.15.33:0/4148984623 learned_addr learned my addr 172.21.15.33:0/4148984623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 0x5650f86778a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc547fff640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a9b0 0x5650f8677de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 -- 172.21.15.33:0/4148984623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a9b0 msgr2=0x5650f8677de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a9b0 0x5650f8677de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 -- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 msgr2=0x5650f86778a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 0x5650f86778a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 -- 172.21.15.33:0/4148984623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53800e040 con 0x5650f8613780 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 0x5650f86778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:10:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc54d224640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5400bc4c0 tx=0x7fc54003ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22a8cc546588589 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc527fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400042d0 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fc5400bc9f0 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.323+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5400bc350 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54003cc20 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 10 client.15618 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5650f8532a80 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc5400bc350 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc5400bdd20 con 0x5650f8613780 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 10 client.15618 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 20 client.15618 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:30.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f867f490 0x5650f86818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 4 client.15618 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:10:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5650f8271600 con 0x5650f867f490 2024-01-25T16:10:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.324+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f867f490 0x5650f86818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.325+0000 7fc527fff640 10 client.15618 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:30.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.325+0000 7fc54ca23640 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f867f490 0x5650f86818b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d42725eadbbba109 server_cookie=ff8ce2fff6e67644 in_seq=0 out_seq=0 2024-01-25T16:10:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc527fff640 1 -- 172.21.15.33:0/4148984623 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5650f8271600 con 0x5650f867f490 2024-01-25T16:10:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc527fff640 10 client.15618 handle_command_reply: tid=0 2024-01-25T16:10:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 client.15618 shutdown 2024-01-25T16:10:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 msgr2=0x5650f8671d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc5400bc4c0 tx=0x7fc54003ce80 comp rx=0 tx=0).stop 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 shutdown_connections 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5650f867f490 0x5650f86818b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650f8613780 0x5650f8671d00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc527fff640 0 client.15618 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5650f860a9b0 0x5650f8677de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 --2- 172.21.15.33:0/4148984623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5650f860a090 0x5650f86778a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 >> 172.21.15.33:0/4148984623 conn(0x5650f86049b0 msgr2=0x5650f8606690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 shutdown_connections 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 1 -- 172.21.15.33:0/4148984623 wait complete. 2024-01-25T16:10:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.367+0000 7fc551846740 20 client.15618 trim_cache size 0 max 16384 2024-01-25T16:10:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:10:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:10:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:10:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:10:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 msgr2=0x7fc5300a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 0x7fc5300a3a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc53c005fd0 tx=0x7fc53c00a710 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 msgr2=0x7fc54819bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc54002c700 tx=0x7fc540035d10 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 shutdown_connections 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fc5300a1570 0x7fc5300a3a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc548104b90 0x7fc54819bb80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc548100800 0x7fc548197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 --2- 172.21.15.33:0/4108830664 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5480ffe40 0x7fc5481972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 >> 172.21.15.33:0/4108830664 conn(0x7fc548076a40 msgr2=0x7fc5480685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 shutdown_connections 2024-01-25T16:10:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:30.368+0000 7fc54ec87640 1 -- 172.21.15.33:0/4108830664 wait complete. 2024-01-25T16:10:30.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45027 2.96 MB/sec warmup 87 sec latency 4962.236 ms 2024-01-25T16:10:31.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:31 smithi076 ceph-mon[35642]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 538 KiB/s wr, 100 op/s 2024-01-25T16:10:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:31 smithi033 ceph-mon[31417]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 538 KiB/s wr, 100 op/s 2024-01-25T16:10:31.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:31 smithi152 ceph-mon[36015]: pgmap v83: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 538 KiB/s wr, 100 op/s 2024-01-25T16:10:31.759 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45027 2.93 MB/sec warmup 88 sec latency 1291.668 ms 2024-01-25T16:10:32.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45027 2.89 MB/sec warmup 89 sec latency 2291.792 ms 2024-01-25T16:10:33.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:33 smithi076 ceph-mon[35642]: pgmap v84: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 832 KiB/s wr, 134 op/s 2024-01-25T16:10:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:33 smithi033 ceph-mon[31417]: pgmap v84: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 832 KiB/s wr, 134 op/s 2024-01-25T16:10:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:33 smithi152 ceph-mon[36015]: pgmap v84: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 832 KiB/s wr, 134 op/s 2024-01-25T16:10:33.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45027 2.86 MB/sec warmup 90 sec latency 3291.889 ms 2024-01-25T16:10:34.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45027 2.83 MB/sec warmup 91 sec latency 4291.986 ms 2024-01-25T16:10:35.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:35 smithi076 ceph-mon[35642]: pgmap v85: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:35 smithi033 ceph-mon[31417]: pgmap v85: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:35 smithi152 ceph-mon[36015]: pgmap v85: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:35.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 45917 2.88 MB/sec warmup 92 sec latency 4968.470 ms 2024-01-25T16:10:36.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48443 2.97 MB/sec warmup 93 sec latency 12.662 ms 2024-01-25T16:10:37.420 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:37 smithi076 ceph-mon[35642]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:37 smithi033 ceph-mon[31417]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:37.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:37 smithi152 ceph-mon[36015]: pgmap v86: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 598 KiB/s wr, 75 op/s 2024-01-25T16:10:37.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48482 2.94 MB/sec warmup 94 sec latency 984.731 ms 2024-01-25T16:10:38.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48482 2.91 MB/sec warmup 95 sec latency 1984.832 ms 2024-01-25T16:10:39.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:39 smithi076 ceph-mon[35642]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 972 KiB/s wr, 96 op/s 2024-01-25T16:10:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:39 smithi033 ceph-mon[31417]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 972 KiB/s wr, 96 op/s 2024-01-25T16:10:39.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:39 smithi152 ceph-mon[36015]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 972 KiB/s wr, 96 op/s 2024-01-25T16:10:39.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48482 2.88 MB/sec warmup 96 sec latency 2984.951 ms 2024-01-25T16:10:40.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48576 2.85 MB/sec warmup 97 sec latency 3668.140 ms 2024-01-25T16:10:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:41 smithi076 ceph-mon[35642]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 669 KiB/s wr, 55 op/s 2024-01-25T16:10:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:41 smithi033 ceph-mon[31417]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 669 KiB/s wr, 55 op/s 2024-01-25T16:10:41.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:41 smithi152 ceph-mon[36015]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 669 KiB/s wr, 55 op/s 2024-01-25T16:10:41.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48576 2.82 MB/sec warmup 98 sec latency 1236.047 ms 2024-01-25T16:10:42.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48576 2.79 MB/sec warmup 99 sec latency 2236.167 ms 2024-01-25T16:10:43.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:43 smithi076 ceph-mon[35642]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-01-25T16:10:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 -- 172.21.15.33:0/3100413144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c1018e0 msgr2=0x7fd63c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 --2- 172.21.15.33:0/3100413144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c1018e0 0x7fd63c105920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd630009930 tx=0x7fd63002bc30 comp rx=0 tx=0).stop 2024-01-25T16:10:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 -- 172.21.15.33:0/3100413144 shutdown_connections 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 --2- 172.21.15.33:0/3100413144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c10dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 --2- 172.21.15.33:0/3100413144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c1018e0 0x7fd63c105920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 --2- 172.21.15.33:0/3100413144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 0x7fd63c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 -- 172.21.15.33:0/3100413144 >> 172.21.15.33:0/3100413144 conn(0x7fd63c0fc7d0 msgr2=0x7fd63c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 -- 172.21.15.33:0/3100413144 shutdown_connections 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.466+0000 7fd642241640 1 -- 172.21.15.33:0/3100413144 wait complete. 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 Processor -- start 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 -- start start 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 0x7fd63c197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19c480 con 0x7fd63c19ba90 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19c5f0 con 0x7fd63c106000 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd642241640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c19c760 con 0x7fd63c100f30 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd63bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd63bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36314/0 (socket says 172.21.15.33:36314) 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd63bfff640 1 -- 172.21.15.33:0/2835964102 learned_addr learned my addr 172.21.15.33:0/2835964102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.467+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 0x7fd63c197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63affd640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 -- 172.21.15.33:0/2835964102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 msgr2=0x7fd63c197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 0x7fd63c197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 -- 172.21.15.33:0/2835964102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 msgr2=0x7fd63c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 -- 172.21.15.33:0/2835964102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009560 con 0x7fd63c19ba90 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63affd640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:43.473 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd63bfff640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd63400eb10 tx=0x7fd63400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd638ff9640 1 -- 172.21.15.33:0/2835964102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004b20 con 0x7fd63c19ba90 2024-01-25T16:10:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd638ff9640 1 -- 172.21.15.33:0/2835964102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd634004c80 con 0x7fd63c19ba90 2024-01-25T16:10:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c1a3d40 con 0x7fd63c19ba90 2024-01-25T16:10:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd638ff9640 1 -- 172.21.15.33:0/2835964102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634010690 con 0x7fd63c19ba90 2024-01-25T16:10:43.474 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.468+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1a41f0 con 0x7fd63c19ba90 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.470+0000 7fd638ff9640 1 -- 172.21.15.33:0/2835964102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd634005020 con 0x7fd63c19ba90 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 Processor -- start 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 -- start start 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bed470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.480 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3b5c070 con 0x55b9a3bf6780 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3b59760 con 0x55b9a3bed090 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd644e07740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3b55050 con 0x55b9a3bed9b0 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd638ff9640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 0x7fd6240a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36326/0 (socket says 172.21.15.33:36326) 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63affd640 1 -- 172.21.15.33:0/3193441371 learned_addr learned my addr 172.21.15.33:0/3193441371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd638ff9640 1 -- 172.21.15.33:0/2835964102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd634014070 con 0x7fd63c19ba90 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63affd640 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bed470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63bfff640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 0x7fd6240a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.471+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 -- 172.21.15.33:0/3193441371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 msgr2=0x55b9a3bf6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 -- 172.21.15.33:0/3193441371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 msgr2=0x55b9a3bed470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bed470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 -- 172.21.15.33:0/3193441371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a3b4ce70 con 0x55b9a3bf6780 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63affd640 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd630009930 tx=0x7fd630000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5ba9938647e903 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd63bfff640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 0x7fd6240a3a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6340049d0 tx=0x7fd634009a70 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd621ffb640 1 -- 172.21.15.33:0/3193441371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63002c980 con 0x55b9a3bf6780 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd621ffb640 1 -- 172.21.15.33:0/3193441371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630004af0 con 0x55b9a3bf6780 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.472+0000 7fd621ffb640 1 -- 172.21.15.33:0/3193441371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6300054e0 con 0x55b9a3bf6780 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- 172.21.15.33:0/3193441371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 msgr2=0x55b9a3bf8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd630009930 tx=0x7fd630000bc0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- 172.21.15.33:0/3193441371 shutdown_connections 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf8b70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 --2- 172.21.15.33:0/3193441371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bed470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- 172.21.15.33:0/3193441371 >> 172.21.15.33:0/3193441371 conn(0x55b9a3be79b0 msgr2=0x55b9a3be7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- 172.21.15.33:0/3193441371 shutdown_connections 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- 172.21.15.33:0/3193441371 wait complete. 2024-01-25T16:10:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 -- start start 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.473+0000 7fd644e07740 1 Processor -- start 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bf3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3b0e1e0 con 0x55b9a3bf6780 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3b0de80 con 0x55b9a3bed090 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3ad30c0 con 0x55b9a3bed9b0 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 10 client.0 _ll_register_callbacks cb 0x55b9a3af8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 -- 172.21.15.33:0/3872476241 learned_addr learned my addr 172.21.15.33:0/3872476241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bf3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 -- 172.21.15.33:0/3872476241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 msgr2=0x55b9a3bf3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 -- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 msgr2=0x55b9a3bf3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bf3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 -- 172.21.15.33:0/3872476241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c10e3e0 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd63affd640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd630004f80 tx=0x7fd630045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455d653057c81d9f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd61affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630004450 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd63002c780 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.474+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b9a3b4e040 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd63002c980 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 10 client.15630 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b9a3b536d0 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b9a3b4e040 con 0x55b9a3bf6780 2024-01-25T16:10:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd630043450 con 0x55b9a3bf6780 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 10 client.15630 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 20 client.15630 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c06280 0x55b9a3c086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 4 client.15630 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9a3b4dd60 con 0x55b9a3c06280 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.475+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c06280 0x55b9a3c086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.476+0000 7fd61affd640 10 client.15630 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.476+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c06280 0x55b9a3c086a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3add8ee8642d0863 server_cookie=20edccf7b4a08b1e in_seq=0 out_seq=0 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.476+0000 7fd61affd640 1 -- 172.21.15.33:0/3872476241 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9a3b4dd60 con 0x55b9a3c06280 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.476+0000 7fd61affd640 10 client.15630 handle_command_reply: tid=0 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 client.15630 shutdown 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 msgr2=0x55b9a3bf41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd630004f80 tx=0x7fd630045040 comp rx=0 tx=0).stop 2024-01-25T16:10:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 shutdown_connections 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c06280 0x55b9a3c086a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bf6780 0x55b9a3bf41b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bf3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 --2- 172.21.15.33:0/3872476241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed090 0x55b9a3bf3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd61affd640 0 client.15630 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 >> 172.21.15.33:0/3872476241 conn(0x55b9a3be79b0 msgr2=0x55b9a3be9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 shutdown_connections 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 1 -- 172.21.15.33:0/3872476241 wait complete. 2024-01-25T16:10:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.477+0000 7fd644e07740 20 client.15630 trim_cache size 0 max 16384 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.490+0000 7fd644e07740 1 Processor -- start 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.490+0000 7fd644e07740 1 -- start start 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3beea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3beef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3be30e0 con 0x55b9a3bed090 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3c03020 con 0x55b9a3bed9b0 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd644e07740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3c06080 con 0x55b9a3bf6780 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36344/0 (socket says 172.21.15.33:36344) 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 -- 172.21.15.33:0/2536616180 learned_addr learned my addr 172.21.15.33:0/2536616180 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:43.495 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3beea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 -- 172.21.15.33:0/2536616180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 msgr2=0x55b9a3beef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3beef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 -- 172.21.15.33:0/2536616180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 msgr2=0x55b9a3beea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3beea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 -- 172.21.15.33:0/2536616180 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9a3af0060 con 0x55b9a3bed090 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd63affd640 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd630001eb0 tx=0x7fd6300058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6589ee19eeb7d1ea server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd6217fa640 1 -- 172.21.15.33:0/2536616180 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630004fb0 con 0x55b9a3bed090 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd6217fa640 1 -- 172.21.15.33:0/2536616180 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd630054070 con 0x55b9a3bed090 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.491+0000 7fd6217fa640 1 -- 172.21.15.33:0/2536616180 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630005730 con 0x55b9a3bed090 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- 172.21.15.33:0/2536616180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 msgr2=0x55b9a3bf5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd630001eb0 tx=0x7fd6300058d0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- 172.21.15.33:0/2536616180 shutdown_connections 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3beef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.496 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3beea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 --2- 172.21.15.33:0/2536616180 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bf5e60 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- 172.21.15.33:0/2536616180 >> 172.21.15.33:0/2536616180 conn(0x55b9a3be79b0 msgr2=0x55b9a3bedf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- 172.21.15.33:0/2536616180 shutdown_connections 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- 172.21.15.33:0/2536616180 wait complete. 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 -- start start 2024-01-25T16:10:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.492+0000 7fd644e07740 1 Processor -- start 2024-01-25T16:10:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bfbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3bfc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3bfc9f0 con 0x55b9a3bed090 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3c5c750 con 0x55b9a3bed9b0 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9a3c5c890 con 0x55b9a3bf6780 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 10 client.0 _ll_register_callbacks cb 0x7fd6340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36358/0 (socket says 172.21.15.33:36358) 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bfbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 -- 172.21.15.33:0/1470590675 learned_addr learned my addr 172.21.15.33:0/1470590675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 -- 172.21.15.33:0/1470590675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 msgr2=0x55b9a3bfc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3bfc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 -- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 msgr2=0x55b9a3bfbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bfbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 -- 172.21.15.33:0/1470590675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9a3b536d0 con 0x55b9a3bed090 2024-01-25T16:10:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd63affd640 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd63002c720 tx=0x7fd630043bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa61d724716cec6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd61affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630043450 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fd630000e40 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.493+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd630004af0 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6300435b0 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 10 client.15636 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b9a3b15a80 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd63007b080 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd630003860 con 0x55b9a3bed090 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 10 client.15636 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 10 client.15636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 20 client.15636 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c5db80 0x55b9a3c5ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 4 client.15636 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:10:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b9a3854600 con 0x55b9a3c5db80 2024-01-25T16:10:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.494+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c5db80 0x55b9a3c5ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.495+0000 7fd61affd640 10 client.15636 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:43.502 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.495+0000 7fd63b7fe640 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c5db80 0x55b9a3c5ff70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61afb22924f6274c server_cookie=3dcececfb19cfad3 in_seq=0 out_seq=0 2024-01-25T16:10:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd61affd640 1 -- 172.21.15.33:0/1470590675 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b9a3854600 con 0x55b9a3c5db80 2024-01-25T16:10:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd61affd640 10 client.15636 handle_command_reply: tid=0 2024-01-25T16:10:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 client.15636 shutdown 2024-01-25T16:10:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 msgr2=0x55b9a3bfd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd63002c720 tx=0x7fd630043bb0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 shutdown_connections 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b9a3c5db80 0x55b9a3c5ff70 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b9a3bf6780 0x55b9a3bfc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9a3bed9b0 0x55b9a3bfbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd61affd640 0 client.15636 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 --2- 172.21.15.33:0/1470590675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b9a3bed090 0x55b9a3bfd880 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 >> 172.21.15.33:0/1470590675 conn(0x55b9a3be79b0 msgr2=0x55b9a3beea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 shutdown_connections 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.532+0000 7fd644e07740 1 -- 172.21.15.33:0/1470590675 wait complete. 2024-01-25T16:10:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd644e07740 20 client.15636 trim_cache size 0 max 16384 2024-01-25T16:10:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:10:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:10:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:10:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:10:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 msgr2=0x7fd6240a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 0x7fd6240a3a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6340049d0 tx=0x7fd634009a70 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 msgr2=0x7fd63c19bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd63400eb10 tx=0x7fd63400c2d0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 shutdown_connections 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd6240a1570 0x7fd6240a3a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd63c19ba90 0x7fd63c19bf40 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c106000 0x7fd63c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 --2- 172.21.15.33:0/2835964102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd63c100f30 0x7fd63c197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 >> 172.21.15.33:0/2835964102 conn(0x7fd63c0fc7d0 msgr2=0x7fd63c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 shutdown_connections 2024-01-25T16:10:43.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:43.533+0000 7fd642241640 1 -- 172.21.15.33:0/2835964102 wait complete. 2024-01-25T16:10:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:43 smithi033 ceph-mon[31417]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-01-25T16:10:43.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:43 smithi152 ceph-mon[36015]: pgmap v89: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-01-25T16:10:43.760 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48576 2.77 MB/sec warmup 100 sec latency 3236.264 ms 2024-01-25T16:10:44.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 48576 2.74 MB/sec warmup 101 sec latency 4236.380 ms 2024-01-25T16:10:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:45 smithi033 ceph-mon[31417]: pgmap v90: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 770 KiB/s wr, 53 op/s 2024-01-25T16:10:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:45 smithi152 ceph-mon[36015]: pgmap v90: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 770 KiB/s wr, 53 op/s 2024-01-25T16:10:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:45 smithi076 ceph-mon[35642]: pgmap v90: 97 pgs: 97 active+clean; 32 MiB data, 534 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 770 KiB/s wr, 53 op/s 2024-01-25T16:10:45.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 49507 2.78 MB/sec warmup 102 sec latency 4907.899 ms 2024-01-25T16:10:46.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52051 2.87 MB/sec warmup 103 sec latency 12.807 ms 2024-01-25T16:10:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:47 smithi033 ceph-mon[31417]: pgmap v91: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 772 KiB/s wr, 55 op/s 2024-01-25T16:10:47.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:47 smithi152 ceph-mon[36015]: pgmap v91: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 772 KiB/s wr, 55 op/s 2024-01-25T16:10:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:47 smithi076 ceph-mon[35642]: pgmap v91: 97 pgs: 97 active+clean; 32 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 772 KiB/s wr, 55 op/s 2024-01-25T16:10:47.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52080 2.84 MB/sec warmup 104 sec latency 985.017 ms 2024-01-25T16:10:48.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52080 2.82 MB/sec warmup 105 sec latency 1985.102 ms 2024-01-25T16:10:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:49 smithi033 ceph-mon[31417]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 909 KiB/s wr, 88 op/s 2024-01-25T16:10:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:49 smithi152 ceph-mon[36015]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 909 KiB/s wr, 88 op/s 2024-01-25T16:10:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:49 smithi076 ceph-mon[35642]: pgmap v92: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 909 KiB/s wr, 88 op/s 2024-01-25T16:10:49.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52080 2.79 MB/sec warmup 106 sec latency 2985.227 ms 2024-01-25T16:10:50.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52181 2.77 MB/sec warmup 107 sec latency 3687.629 ms 2024-01-25T16:10:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:51 smithi033 ceph-mon[31417]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 535 KiB/s wr, 66 op/s 2024-01-25T16:10:51.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:51 smithi152 ceph-mon[36015]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 535 KiB/s wr, 66 op/s 2024-01-25T16:10:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:51 smithi076 ceph-mon[35642]: pgmap v93: 97 pgs: 97 active+clean; 29 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 535 KiB/s wr, 66 op/s 2024-01-25T16:10:51.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52181 2.74 MB/sec warmup 108 sec latency 1192.492 ms 2024-01-25T16:10:52.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52181 2.72 MB/sec warmup 109 sec latency 2192.589 ms 2024-01-25T16:10:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:53 smithi033 ceph-mon[31417]: pgmap v94: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 940 KiB/s wr, 97 op/s 2024-01-25T16:10:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:53 smithi152 ceph-mon[36015]: pgmap v94: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 940 KiB/s wr, 97 op/s 2024-01-25T16:10:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:53 smithi076 ceph-mon[35642]: pgmap v94: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 940 KiB/s wr, 97 op/s 2024-01-25T16:10:53.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52181 2.69 MB/sec warmup 110 sec latency 3192.712 ms 2024-01-25T16:10:54.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 52181 2.67 MB/sec warmup 111 sec latency 4192.762 ms 2024-01-25T16:10:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:55 smithi033 ceph-mon[31417]: pgmap v95: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 545 KiB/s wr, 65 op/s 2024-01-25T16:10:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:55 smithi152 ceph-mon[36015]: pgmap v95: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 545 KiB/s wr, 65 op/s 2024-01-25T16:10:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:55 smithi076 ceph-mon[35642]: pgmap v95: 97 pgs: 97 active+clean; 30 MiB data, 515 MiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 545 KiB/s wr, 65 op/s 2024-01-25T16:10:55.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 53233 2.73 MB/sec warmup 112 sec latency 4869.497 ms 2024-01-25T16:10:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2194943762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 msgr2=0x7fdd080ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2194943762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd080ff840 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009930 tx=0x7fdcf802bc30 comp rx=0 tx=0).stop 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2194943762 shutdown_connections 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2194943762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 0x7fdd0810dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2194943762 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd080ffe10 0x7fdd08105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2194943762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd080ff840 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2194943762 >> 172.21.15.33:0/2194943762 conn(0x7fdd08076140 msgr2=0x7fdd080fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2194943762 shutdown_connections 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.624+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2194943762 wait complete. 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 Processor -- start 2024-01-25T16:10:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 -- start start 2024-01-25T16:10:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 0x7fdd081977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd0819bad0 0x7fdd0819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.630 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819c4c0 con 0x7fdd080ff460 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819c630 con 0x7fdd08106020 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.625+0000 7fdd0e6b4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0819c7a0 con 0x7fdd0819bad0 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43632/0 (socket says 172.21.15.33:43632) 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 -- 172.21.15.33:0/2570003367 learned_addr learned my addr 172.21.15.33:0/2570003367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd0819bad0 0x7fdd0819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd077fe640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 0x7fdd081977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 -- 172.21.15.33:0/2570003367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd0819bad0 msgr2=0x7fdd0819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd0819bad0 0x7fdd0819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 -- 172.21.15.33:0/2570003367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 msgr2=0x7fdd081977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 0x7fdd081977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 -- 172.21.15.33:0/2570003367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf8009560 con 0x7fdd080ff460 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd07fff640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005940 tx=0x7fdcf8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd057fa640 1 -- 172.21.15.33:0/2570003367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8004e50 con 0x7fdd080ff460 2024-01-25T16:10:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd081a3d80 con 0x7fdd080ff460 2024-01-25T16:10:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd057fa640 1 -- 172.21.15.33:0/2570003367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf80054c0 con 0x7fdd080ff460 2024-01-25T16:10:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd057fa640 1 -- 172.21.15.33:0/2570003367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8003b20 con 0x7fdd080ff460 2024-01-25T16:10:56.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.626+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd081a41b0 con 0x7fdd080ff460 2024-01-25T16:10:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.628+0000 7fdd057fa640 1 -- 172.21.15.33:0/2570003367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fdcf8003c80 con 0x7fdd080ff460 2024-01-25T16:10:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 Processor -- start 2024-01-25T16:10:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 -- start start 2024-01-25T16:10:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd057fa640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 0x7fdcf00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 0x56223c442470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 0x7fdcf00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c44db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd057fa640 1 -- 172.21.15.33:0/2570003367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcf80ef530 con 0x7fdd080ff460 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c3b1070 con 0x56223c4429b0 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c3ae760 con 0x56223c44b780 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd11277740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c3aa050 con 0x56223c442090 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43644/0 (socket says 172.21.15.33:43644) 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 -- 172.21.15.33:0/2503177670 learned_addr learned my addr 172.21.15.33:0/2503177670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c44db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd07fff640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 0x56223c442470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 -- 172.21.15.33:0/2503177670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 msgr2=0x56223c442470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 0x56223c442470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 -- 172.21.15.33:0/2503177670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 msgr2=0x56223c44db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c44db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd077fe640 1 -- 172.21.15.33:0/2503177670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56223c3a1e70 con 0x56223c4429b0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.629+0000 7fdd07fff640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 0x56223c442470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 0x7fdcf00a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd00005fd0 tx=0x7fdd00011b40 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdd077fe640 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009c60 tx=0x7fdcfc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6b3ee924352cf1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdcee7fc640 1 -- 172.21.15.33:0/2503177670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc004250 con 0x56223c4429b0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdcee7fc640 1 -- 172.21.15.33:0/2503177670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcfc009e10 con 0x56223c4429b0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdcee7fc640 1 -- 172.21.15.33:0/2503177670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc019760 con 0x56223c4429b0 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdd11277740 1 -- 172.21.15.33:0/2503177670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 msgr2=0x56223c44b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.630+0000 7fdd11277740 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009c60 tx=0x7fdcfc011810 comp rx=0 tx=0).stop 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- 172.21.15.33:0/2503177670 shutdown_connections 2024-01-25T16:10:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c44db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c44b240 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- 172.21.15.33:0/2503177670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c442090 0x56223c442470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- 172.21.15.33:0/2503177670 >> 172.21.15.33:0/2503177670 conn(0x56223c43c9b0 msgr2=0x56223c43cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- 172.21.15.33:0/2503177670 shutdown_connections 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- 172.21.15.33:0/2503177670 wait complete. 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- start start 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 Processor -- start 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c449190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c3631e0 con 0x56223c442090 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c362e80 con 0x56223c44b780 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c3280c0 con 0x56223c4429b0 2024-01-25T16:10:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.631+0000 7fdd11277740 10 client.0 _ll_register_callbacks cb 0x56223c34dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43652/0 (socket says 172.21.15.33:43652) 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd0cc51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c449190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 -- 172.21.15.33:0/683180331 learned_addr learned my addr 172.21.15.33:0/683180331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd077fe640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 -- 172.21.15.33:0/683180331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 msgr2=0x56223c448c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 -- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 msgr2=0x56223c449190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c449190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 -- 172.21.15.33:0/683180331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc009890 con 0x56223c442090 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd077fe640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c449190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd07fff640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80bc380 tx=0x7fdcf80bcbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb84103b487ef7d4 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80c4780 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcf80042e0 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56223c3a3040 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcf80044a0 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd11277740 10 client.15648 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.632+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56223c3a86d0 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcf80da020 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdcf809c020 con 0x56223c442090 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 10 client.15648 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 20 client.15648 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c45b310 0x56223c45d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 4 client.15648 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56223c3a2d60 con 0x56223c45b310 2024-01-25T16:10:56.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd077fe640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c45b310 0x56223c45d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdd077fe640 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c45b310 0x56223c45d700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4b46c8defd16ca server_cookie=3d5c876696099b04 in_seq=0 out_seq=0 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.633+0000 7fdce77fe640 10 client.15648 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdce77fe640 1 -- 172.21.15.33:0/683180331 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56223c3a2d60 con 0x56223c45b310 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdce77fe640 10 client.15648 handle_command_reply: tid=0 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 client.15648 shutdown 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 msgr2=0x56223c448710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80bc380 tx=0x7fdcf80bcbb0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 shutdown_connections 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c45b310 0x56223c45d700 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c44b780 0x56223c449190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 --2- 172.21.15.33:0/683180331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c442090 0x56223c448710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdce77fe640 0 client.15648 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 >> 172.21.15.33:0/683180331 conn(0x56223c43c9b0 msgr2=0x56223c43e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 shutdown_connections 2024-01-25T16:10:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 1 -- 172.21.15.33:0/683180331 wait complete. 2024-01-25T16:10:56.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.634+0000 7fdd11277740 20 client.15648 trim_cache size 0 max 16384 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 Processor -- start 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 -- start start 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c44ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c4380e0 con 0x56223c44b780 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c43d9b0 con 0x56223c442090 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd11277740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c44db70 con 0x56223c4429b0 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.649+0000 7fdd0cc51640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43656/0 (socket says 172.21.15.33:43656) 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 -- 172.21.15.33:0/1168424251 learned_addr learned my addr 172.21.15.33:0/1168424251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:56.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd07fff640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c44ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd077fe640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 -- 172.21.15.33:0/1168424251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 msgr2=0x56223c448fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 -- 172.21.15.33:0/1168424251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 msgr2=0x56223c44ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c44ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 -- 172.21.15.33:0/1168424251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56223c345060 con 0x56223c44b780 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd07fff640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c44ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd077fe640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000fdd0 tx=0x7fdd0001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19c1c5579a358b6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdcedffb640 1 -- 172.21.15.33:0/1168424251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00003e00 con 0x56223c44b780 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdcedffb640 1 -- 172.21.15.33:0/1168424251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd00004550 con 0x56223c44b780 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.650+0000 7fdcedffb640 1 -- 172.21.15.33:0/1168424251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00020440 con 0x56223c44b780 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- 172.21.15.33:0/1168424251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 msgr2=0x56223c443a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd0000fdd0 tx=0x7fdd0001e000 comp rx=0 tx=0).stop 2024-01-25T16:10:56.655 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- 172.21.15.33:0/1168424251 shutdown_connections 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c44b780 0x56223c443a60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c4429b0 0x56223c448fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 --2- 172.21.15.33:0/1168424251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c44ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- 172.21.15.33:0/1168424251 >> 172.21.15.33:0/1168424251 conn(0x56223c43c9b0 msgr2=0x56223c442f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- 172.21.15.33:0/1168424251 shutdown_connections 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- 172.21.15.33:0/1168424251 wait complete. 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.651+0000 7fdd11277740 1 -- start start 2024-01-25T16:10:56.656 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 Processor -- start 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c44b780 0x56223c451ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c452420 con 0x56223c4429b0 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c452590 con 0x56223c442090 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56223c452750 con 0x56223c44b780 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd11277740 10 client.0 _ll_register_callbacks cb 0x7fdcfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43666/0 (socket says 172.21.15.33:43666) 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 -- 172.21.15.33:0/699407151 learned_addr learned my addr 172.21.15.33:0/699407151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd07fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:44112/0 (socket says 172.21.15.33:44112) 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd0cc51640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c44b780 0x56223c451ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 -- 172.21.15.33:0/699407151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c44b780 msgr2=0x56223c451ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c44b780 0x56223c451ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 -- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 msgr2=0x56223c4a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 -- 172.21.15.33:0/699407151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56223c3a86d0 con 0x56223c4429b0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd07fff640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdd077fe640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009c60 tx=0x7fdcfc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f04f625a1715a0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdce77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.652+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc0038f0 con 0x56223c4429b0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdcfc003a50 con 0x56223c4429b0 2024-01-25T16:10:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdcfc009e10 con 0x56223c4429b0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcfc003c50 con 0x56223c4429b0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 10 client.15654 fetch_fsmap learned FSMap version 24 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56223c3a1e70 con 0x56223c4429b0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdcfc03c080 con 0x56223c4429b0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fdcfc019430 con 0x56223c4429b0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 10 client.15654 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 20 client.15654 populate_metadata read hostname 'smithi033' 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c4b2b50 0x56223c4b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 4 client.15654 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56223c0a9600 con 0x56223c4b2b50 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.653+0000 7fdd07fff640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c4b2b50 0x56223c4b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.654+0000 7fdd07fff640 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c4b2b50 0x56223c4b4f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371b335a5924e37c server_cookie=79611820456b5fa5 in_seq=0 out_seq=0 2024-01-25T16:10:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.654+0000 7fdce77fe640 10 client.15654 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdce77fe640 1 -- 172.21.15.33:0/699407151 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56223c0a9600 con 0x56223c4b2b50 2024-01-25T16:10:56.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdce77fe640 10 client.15654 handle_command_reply: tid=0 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 client.15654 shutdown 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 msgr2=0x56223c4a9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc009c60 tx=0x7fdcfc005270 comp rx=0 tx=0).stop 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 shutdown_connections 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56223c4b2b50 0x56223c4b4f70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56223c44b780 0x56223c451ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdce77fe640 0 client.15654 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56223c4429b0 0x56223c4a9f50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 --2- 172.21.15.33:0/699407151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56223c442090 0x56223c4a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 >> 172.21.15.33:0/699407151 conn(0x56223c43c9b0 msgr2=0x56223c443a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 shutdown_connections 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.710+0000 7fdd11277740 1 -- 172.21.15.33:0/699407151 wait complete. 2024-01-25T16:10:56.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd11277740 20 client.15654 trim_cache size 0 max 16384 2024-01-25T16:10:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:10:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:10:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:10:56.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 msgr2=0x7fdcf00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 0x7fdcf00a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd00005fd0 tx=0x7fdd00011b40 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 msgr2=0x7fdd08197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005940 tx=0x7fdcf8005d50 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 shutdown_connections 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdcf00a14f0 0x7fdcf00a39b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdd0819bad0 0x7fdd0819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd08106020 0x7fdd081977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 --2- 172.21.15.33:0/2570003367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd080ff460 0x7fdd08197260 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:10:56.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 >> 172.21.15.33:0/2570003367 conn(0x7fdd08076140 msgr2=0x7fdd081041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:10:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 shutdown_connections 2024-01-25T16:10:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:10:56.711+0000 7fdd0e6b4640 1 -- 172.21.15.33:0/2570003367 wait complete. 2024-01-25T16:10:56.761 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55572 2.78 MB/sec warmup 113 sec latency 15.063 ms 2024-01-25T16:10:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:57 smithi033 ceph-mon[31417]: pgmap v96: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 623 KiB/s wr, 70 op/s 2024-01-25T16:10:57.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:57 smithi152 ceph-mon[36015]: pgmap v96: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 623 KiB/s wr, 70 op/s 2024-01-25T16:10:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:57 smithi076 ceph-mon[35642]: pgmap v96: 97 pgs: 97 active+clean; 30 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 623 KiB/s wr, 70 op/s 2024-01-25T16:10:57.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55626 2.76 MB/sec warmup 114 sec latency 972.545 ms 2024-01-25T16:10:58.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:58 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:10:58.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:58 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:10:58.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:58 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:10:58.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55626 2.74 MB/sec warmup 115 sec latency 1972.660 ms 2024-01-25T16:10:59.534 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:59 smithi152 ceph-mon[36015]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 995 KiB/s wr, 88 op/s 2024-01-25T16:10:59.534 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:10:59.534 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:10:59.534 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:10:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:10:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:59 smithi033 ceph-mon[31417]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 995 KiB/s wr, 88 op/s 2024-01-25T16:10:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:10:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:10:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:10:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:10:59.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:59 smithi076 ceph-mon[35642]: pgmap v97: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 995 KiB/s wr, 88 op/s 2024-01-25T16:10:59.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:10:59.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:10:59.569 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:10:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:10:59.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 55626 2.71 MB/sec warmup 116 sec latency 2972.773 ms 2024-01-25T16:11:00.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 56424 2.74 MB/sec warmup 117 sec latency 3654.624 ms 2024-01-25T16:11:01.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:01 smithi076 ceph-mon[35642]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 858 KiB/s wr, 56 op/s 2024-01-25T16:11:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:01 smithi033 ceph-mon[31417]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 858 KiB/s wr, 56 op/s 2024-01-25T16:11:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:01 smithi152 ceph-mon[36015]: pgmap v98: 97 pgs: 97 active+clean; 30 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 858 KiB/s wr, 56 op/s 2024-01-25T16:11:01.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 57693 2.78 MB/sec warmup 118 sec latency 23.097 ms 2024-01-25T16:11:02.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59115 2.80 MB/sec warmup 119 sec latency 632.490 ms 2024-01-25T16:11:03.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:03 smithi076 ceph-mon[35642]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-01-25T16:11:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:03 smithi033 ceph-mon[31417]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-01-25T16:11:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:03 smithi152 ceph-mon[36015]: pgmap v99: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-01-25T16:11:04.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59115 0.00 MB/sec execute 1 sec latency 2632.656 ms 2024-01-25T16:11:05.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:05 smithi076 ceph-mon[35642]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 770 KiB/s wr, 121 op/s 2024-01-25T16:11:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:05 smithi033 ceph-mon[31417]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 770 KiB/s wr, 121 op/s 2024-01-25T16:11:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:05 smithi152 ceph-mon[36015]: pgmap v100: 97 pgs: 97 active+clean; 31 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 770 KiB/s wr, 121 op/s 2024-01-25T16:11:05.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59270 0.44 MB/sec execute 2 sec latency 3299.445 ms 2024-01-25T16:11:06.762 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59270 0.30 MB/sec execute 3 sec latency 1150.859 ms 2024-01-25T16:11:07.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:07 smithi076 ceph-mon[35642]: pgmap v101: 97 pgs: 97 active+clean; 33 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 910 KiB/s wr, 138 op/s 2024-01-25T16:11:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:07 smithi033 ceph-mon[31417]: pgmap v101: 97 pgs: 97 active+clean; 33 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 910 KiB/s wr, 138 op/s 2024-01-25T16:11:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:07 smithi152 ceph-mon[36015]: pgmap v101: 97 pgs: 97 active+clean; 33 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 910 KiB/s wr, 138 op/s 2024-01-25T16:11:07.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59270 0.22 MB/sec execute 4 sec latency 2150.959 ms 2024-01-25T16:11:08.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59270 0.18 MB/sec execute 5 sec latency 3151.101 ms 2024-01-25T16:11:09.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:09 smithi076 ceph-mon[35642]: pgmap v102: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 924 KiB/s wr, 148 op/s 2024-01-25T16:11:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:09 smithi033 ceph-mon[31417]: pgmap v102: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 924 KiB/s wr, 148 op/s 2024-01-25T16:11:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:09 smithi152 ceph-mon[36015]: pgmap v102: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 924 KiB/s wr, 148 op/s 2024-01-25T16:11:09.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 59270 0.15 MB/sec execute 6 sec latency 4151.156 ms 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 -- 172.21.15.33:0/3535623687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 msgr2=0x7f335810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 --2- 172.21.15.33:0/3535623687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f335810dbd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3350009cd0 tx=0x7f335002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 -- 172.21.15.33:0/3535623687 shutdown_connections 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 --2- 172.21.15.33:0/3535623687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f335810dbd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 --2- 172.21.15.33:0/3535623687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33581018e0 0x7f3358105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 --2- 172.21.15.33:0/3535623687 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 0x7f3358101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 -- 172.21.15.33:0/3535623687 >> 172.21.15.33:0/3535623687 conn(0x7f33580fc7d0 msgr2=0x7f33580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 -- 172.21.15.33:0/3535623687 shutdown_connections 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.808+0000 7f335f1a4640 1 -- 172.21.15.33:0/3535623687 wait complete. 2024-01-25T16:11:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 Processor -- start 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 -- start start 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 0x7f33581971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335819ba60 0x7f335819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335819c450 con 0x7f3358106000 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335819c5c0 con 0x7f335819ba60 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335f1a4640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335819c730 con 0x7f3358100f30 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50852/0 (socket says 172.21.15.33:50852) 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 -- 172.21.15.33:0/185416231 learned_addr learned my addr 172.21.15.33:0/185416231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335d741640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335819ba60 0x7f335819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f335cf40640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 0x7f33581971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 -- 172.21.15.33:0/185416231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 msgr2=0x7f33581971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 0x7f33581971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 -- 172.21.15.33:0/185416231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335819ba60 msgr2=0x7f335819bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335819ba60 0x7f335819bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.809+0000 7f3357fff640 1 -- 172.21.15.33:0/185416231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350009900 con 0x7f3358106000 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f3357fff640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3344005fd0 tx=0x7f334400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f3355ffb640 1 -- 172.21.15.33:0/185416231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344009c30 con 0x7f3358106000 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f3355ffb640 1 -- 172.21.15.33:0/185416231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344009d90 con 0x7f3358106000 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33581a3d10 con 0x7f3358106000 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f3355ffb640 1 -- 172.21.15.33:0/185416231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344003e80 con 0x7f3358106000 2024-01-25T16:11:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.810+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33581a41c0 con 0x7f3358106000 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.812+0000 7f3355ffb640 1 -- 172.21.15.33:0/185416231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3344003fe0 con 0x7f3358106000 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 Processor -- start 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 -- start start 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3355ffb640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 0x7f33480a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e2af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d9a070 con 0x560fd6e38b90 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d97760 con 0x560fd6e2b4d0 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3361d67740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d93050 con 0x560fd6e2abb0 2024-01-25T16:11:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 0x7f33480a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3355ffb640 1 -- 172.21.15.33:0/185416231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f33440ccb50 con 0x7f3358106000 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50864/0 (socket says 172.21.15.33:50864) 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 -- 172.21.15.33:0/3516909594 learned_addr learned my addr 172.21.15.33:0/3516909594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335cf40640 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e2af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f3357fff640 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 -- 172.21.15.33:0/3516909594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 msgr2=0x560fd6e2af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e2af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 -- 172.21.15.33:0/3516909594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 msgr2=0x560fd6e38650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e38650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.813+0000 7f335d741640 1 -- 172.21.15.33:0/3516909594 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd6d8ae70 con 0x560fd6e38b90 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f335d741640 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3350009430 tx=0x7f3350034860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d860467dd3455c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f335d741640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 0x7f33480a3a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3350004af0 tx=0x7f335003b000 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f3342ffd640 1 -- 172.21.15.33:0/3516909594 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350048070 con 0x560fd6e38b90 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f3342ffd640 1 -- 172.21.15.33:0/3516909594 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3350032b20 con 0x560fd6e38b90 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f3342ffd640 1 -- 172.21.15.33:0/3516909594 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350003d20 con 0x560fd6e38b90 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f3361d67740 1 -- 172.21.15.33:0/3516909594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 msgr2=0x560fd6e3af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.814+0000 7f3361d67740 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3350009430 tx=0x7f3350034860 comp rx=0 tx=0).stop 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 -- 172.21.15.33:0/3516909594 shutdown_connections 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e3af80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e38650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- 172.21.15.33:0/3516909594 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e2af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 -- 172.21.15.33:0/3516909594 >> 172.21.15.33:0/3516909594 conn(0x560fd6e259b0 msgr2=0x560fd6e25d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 -- 172.21.15.33:0/3516909594 shutdown_connections 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 -- 172.21.15.33:0/3516909594 wait complete. 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 -- start start 2024-01-25T16:11:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 Processor -- start 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e35ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.815+0000 7f3361d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3361d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d4c1e0 con 0x560fd6e38b90 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3361d67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d4be80 con 0x560fd6e2b4d0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3361d67740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6d110c0 con 0x560fd6e2abb0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3361d67740 10 client.0 _ll_register_callbacks cb 0x560fd6d36ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50880/0 (socket says 172.21.15.33:50880) 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 -- 172.21.15.33:0/1847546580 learned_addr learned my addr 172.21.15.33:0/1847546580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3357fff640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e35ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335cf40640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 -- 172.21.15.33:0/1847546580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 msgr2=0x560fd6e35ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e35ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 -- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 msgr2=0x560fd6e35ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e35ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 -- 172.21.15.33:0/1847546580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335810e3e0 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3357fff640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f335d741640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3350003d70 tx=0x7f3350003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413f622d4b90b188 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3337fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350048070 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f3350035d80 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fd6d8c040 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.816+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3350032340 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 10 client.15666 fetch_fsmap learned FSMap version 24 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560fd6d916d0 con 0x560fd6e38b90 2024-01-25T16:11:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3350062020 con 0x560fd6e38b90 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f33500425f0 con 0x560fd6e38b90 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 10 client.15666 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 20 client.15666 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6e48600 0x560fd6e4a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 4 client.15666 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fd6d8bd60 con 0x560fd6e48600 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.817+0000 7f335cf40640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6e48600 0x560fd6e4a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.818+0000 7f3337fff640 10 client.15666 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.818+0000 7f335cf40640 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6e48600 0x560fd6e4a9f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7394558a50e1465 server_cookie=dc40f877e9f0ad1a in_seq=0 out_seq=0 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.818+0000 7f3337fff640 1 -- 172.21.15.33:0/1847546580 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560fd6d8bd60 con 0x560fd6e48600 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.818+0000 7f3337fff640 10 client.15666 handle_command_reply: tid=0 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.818+0000 7f3361d67740 1 client.15666 shutdown 2024-01-25T16:11:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 msgr2=0x560fd6e36530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3350003d70 tx=0x7f3350003da0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 shutdown_connections 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6e48600 0x560fd6e4a9f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e38b90 0x560fd6e36530 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e35ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 --2- 172.21.15.33:0/1847546580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e35ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3337fff640 0 client.15666 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 >> 172.21.15.33:0/1847546580 conn(0x560fd6e259b0 msgr2=0x560fd6e27700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 shutdown_connections 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 1 -- 172.21.15.33:0/1847546580 wait complete. 2024-01-25T16:11:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.819+0000 7f3361d67740 20 client.15666 trim_cache size 0 max 16384 2024-01-25T16:11:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 Processor -- start 2024-01-25T16:11:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 -- start start 2024-01-25T16:11:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 0x560fd6e32d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e210e0 con 0x560fd6e2abb0 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e21270 con 0x560fd6e2b4d0 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f3361d67740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e38580 con 0x560fd6e38b90 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f335cf40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f335cf40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50882/0 (socket says 172.21.15.33:50882) 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.832+0000 7f335cf40640 1 -- 172.21.15.33:0/754554504 learned_addr learned my addr 172.21.15.33:0/754554504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f3357fff640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 -- 172.21.15.33:0/754554504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 msgr2=0x560fd6e32d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335d741640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 0x560fd6e32d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 0x560fd6e32d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 -- 172.21.15.33:0/754554504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 msgr2=0x560fd6e36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 -- 172.21.15.33:0/754554504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd6d2e060 con 0x560fd6e2abb0 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335d741640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 0x560fd6e32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f335cf40640 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x560fd6e4b300 tx=0x7f334c013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a52680da722fa7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f33427fc640 1 -- 172.21.15.33:0/754554504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c017040 con 0x560fd6e2abb0 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f33427fc640 1 -- 172.21.15.33:0/754554504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334c020020 con 0x560fd6e2abb0 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.833+0000 7f33427fc640 1 -- 172.21.15.33:0/754554504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c0035f0 con 0x560fd6e2abb0 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- 172.21.15.33:0/754554504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 msgr2=0x560fd6e34a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x560fd6e4b300 tx=0x7f334c013830 comp rx=0 tx=0).stop 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- 172.21.15.33:0/754554504 shutdown_connections 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e38b90 0x560fd6e32d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e2b4d0 0x560fd6e36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- 172.21.15.33:0/754554504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2abb0 0x560fd6e34a90 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- 172.21.15.33:0/754554504 >> 172.21.15.33:0/754554504 conn(0x560fd6e259b0 msgr2=0x560fd6e34380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- 172.21.15.33:0/754554504 shutdown_connections 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- 172.21.15.33:0/754554504 wait complete. 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- start start 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 Processor -- start 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e3fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e38b90 0x560fd6e3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e3ee60 con 0x560fd6e2b4d0 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e9eae0 con 0x560fd6e38b90 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.834+0000 7f3361d67740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd6e9ec20 con 0x560fd6e2abb0 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3361d67740 10 client.0 _ll_register_callbacks cb 0x7f3350009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50898/0 (socket says 172.21.15.33:50898) 2024-01-25T16:11:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 -- 172.21.15.33:0/3382290085 learned_addr learned my addr 172.21.15.33:0/3382290085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f335d741640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e38b90 0x560fd6e3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f335cf40640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e3fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 -- 172.21.15.33:0/3382290085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 msgr2=0x560fd6e3fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e3fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 -- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e38b90 msgr2=0x560fd6e3e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e38b90 0x560fd6e3e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 -- 172.21.15.33:0/3382290085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c00b870 con 0x560fd6e2b4d0 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3357fff640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f33440998f0 tx=0x7f3344003660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d51a8e5a33116a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3337fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344099960 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f334409dab0 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3350032b20 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.835+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33440a1cf0 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 10 client.15672 fetch_fsmap learned FSMap version 24 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x560fd6d53a80 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3350032b20 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 24) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3344022020 con 0x560fd6e2b4d0 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 10 client.15672 fetch_fsmap finished waiting for FSMap version 24 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 20 client.15672 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6ea0460 0x560fd6ea2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 4 client.15672 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:11:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560fd6a92600 con 0x560fd6ea0460 2024-01-25T16:11:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.836+0000 7f335cf40640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6ea0460 0x560fd6ea2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.837+0000 7f3337fff640 10 client.15672 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:09.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.837+0000 7f335cf40640 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6ea0460 0x560fd6ea2880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe8046f3358222d server_cookie=43e49b1e9b76be8e in_seq=0 out_seq=0 2024-01-25T16:11:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3337fff640 1 -- 172.21.15.33:0/3382290085 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560fd6a92600 con 0x560fd6ea0460 2024-01-25T16:11:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3337fff640 10 client.15672 handle_command_reply: tid=0 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 client.15672 shutdown 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 msgr2=0x560fd6e3e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f33440998f0 tx=0x7f3344003660 comp rx=0 tx=0).stop 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 shutdown_connections 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560fd6ea0460 0x560fd6ea2880 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fd6e38b90 0x560fd6e3e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560fd6e2b4d0 0x560fd6e3e050 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3337fff640 0 client.15672 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 --2- 172.21.15.33:0/3382290085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560fd6e2abb0 0x560fd6e3fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 >> 172.21.15.33:0/3382290085 conn(0x560fd6e259b0 msgr2=0x560fd6e24a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.883+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 shutdown_connections 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f3361d67740 1 -- 172.21.15.33:0/3382290085 wait complete. 2024-01-25T16:11:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f3361d67740 20 client.15672 trim_cache size 0 max 16384 2024-01-25T16:11:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:11:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:11:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:11:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 msgr2=0x7f33480a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 0x7f33480a3a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3350004af0 tx=0x7f335003b000 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 msgr2=0x7f33581976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3344005fd0 tx=0x7f334400dde0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 shutdown_connections 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f33480a1570 0x7f33480a3a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f335819ba60 0x7f335819bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3358106000 0x7f33581976e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 --2- 172.21.15.33:0/185416231 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3358100f30 0x7f33581971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 >> 172.21.15.33:0/185416231 conn(0x7f33580fc7d0 msgr2=0x7f33580feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 shutdown_connections 2024-01-25T16:11:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:09.884+0000 7f335f1a4640 1 -- 172.21.15.33:0/185416231 wait complete. 2024-01-25T16:11:10.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 60307 1.51 MB/sec execute 7 sec latency 4820.996 ms 2024-01-25T16:11:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:11 smithi076 ceph-mon[35642]: pgmap v103: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 549 KiB/s wr, 128 op/s 2024-01-25T16:11:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:11 smithi033 ceph-mon[31417]: pgmap v103: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 549 KiB/s wr, 128 op/s 2024-01-25T16:11:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:11 smithi152 ceph-mon[36015]: pgmap v103: 97 pgs: 97 active+clean; 33 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 549 KiB/s wr, 128 op/s 2024-01-25T16:11:11.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 62665 2.39 MB/sec execute 8 sec latency 23.297 ms 2024-01-25T16:11:12.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 62665 2.12 MB/sec execute 9 sec latency 1023.381 ms 2024-01-25T16:11:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:13 smithi076 ceph-mon[35642]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 966 KiB/s wr, 171 op/s 2024-01-25T16:11:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:13 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:13 smithi033 ceph-mon[31417]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 966 KiB/s wr, 171 op/s 2024-01-25T16:11:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:13 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:13 smithi152 ceph-mon[36015]: pgmap v104: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 966 KiB/s wr, 171 op/s 2024-01-25T16:11:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:13 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:13.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 62665 1.91 MB/sec execute 10 sec latency 2023.518 ms 2024-01-25T16:11:14.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 62665 1.74 MB/sec execute 11 sec latency 3023.623 ms 2024-01-25T16:11:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:15 smithi033 ceph-mon[31417]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 649 KiB/s wr, 74 op/s 2024-01-25T16:11:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:15 smithi152 ceph-mon[36015]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 649 KiB/s wr, 74 op/s 2024-01-25T16:11:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:15 smithi076 ceph-mon[35642]: pgmap v105: 97 pgs: 97 active+clean; 31 MiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 649 KiB/s wr, 74 op/s 2024-01-25T16:11:15.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 63243 2.06 MB/sec execute 12 sec latency 3693.554 ms 2024-01-25T16:11:16.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 65392 2.81 MB/sec execute 13 sec latency 13.593 ms 2024-01-25T16:11:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:17 smithi033 ceph-mon[31417]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 974 KiB/s wr, 101 op/s 2024-01-25T16:11:17.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:17 smithi152 ceph-mon[36015]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 974 KiB/s wr, 101 op/s 2024-01-25T16:11:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:17 smithi076 ceph-mon[35642]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 974 KiB/s wr, 101 op/s 2024-01-25T16:11:17.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 66206 2.73 MB/sec execute 14 sec latency 850.834 ms 2024-01-25T16:11:18.763 INFO:tasks.workunit.client.0.smithi033.stdout: 1 66206 2.55 MB/sec execute 15 sec latency 1850.922 ms 2024-01-25T16:11:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:19 smithi033 ceph-mon[31417]: pgmap v107: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 918 KiB/s wr, 88 op/s 2024-01-25T16:11:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:19 smithi152 ceph-mon[36015]: pgmap v107: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 918 KiB/s wr, 88 op/s 2024-01-25T16:11:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:19 smithi076 ceph-mon[35642]: pgmap v107: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 918 KiB/s wr, 88 op/s 2024-01-25T16:11:19.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 66206 2.39 MB/sec execute 16 sec latency 2851.070 ms 2024-01-25T16:11:20.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 66811 2.58 MB/sec execute 17 sec latency 3529.659 ms 2024-01-25T16:11:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:21 smithi033 ceph-mon[31417]: pgmap v108: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 825 KiB/s wr, 73 op/s 2024-01-25T16:11:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:21 smithi152 ceph-mon[36015]: pgmap v108: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 825 KiB/s wr, 73 op/s 2024-01-25T16:11:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:21 smithi076 ceph-mon[35642]: pgmap v108: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 825 KiB/s wr, 73 op/s 2024-01-25T16:11:21.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 68939 3.09 MB/sec execute 18 sec latency 8.340 ms 2024-01-25T16:11:22.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 69809 3.03 MB/sec execute 19 sec latency 811.990 ms 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 -- 172.21.15.33:0/246378407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc100f50 msgr2=0x7effcc101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 --2- 172.21.15.33:0/246378407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc100f50 0x7effcc101330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7effb4009930 tx=0x7effb402bc30 comp rx=0 tx=0).stop 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 -- 172.21.15.33:0/246378407 shutdown_connections 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 --2- 172.21.15.33:0/246378407 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc106020 0x7effcc10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 --2- 172.21.15.33:0/246378407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc101900 0x7effcc105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 --2- 172.21.15.33:0/246378407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc100f50 0x7effcc101330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.987 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 -- 172.21.15.33:0/246378407 >> 172.21.15.33:0/246378407 conn(0x7effcc0fc7b0 msgr2=0x7effcc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 -- 172.21.15.33:0/246378407 shutdown_connections 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.982+0000 7effd2cec640 1 -- 172.21.15.33:0/246378407 wait complete. 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 Processor -- start 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 -- start start 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 0x7effcc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc19bb10 0x7effcc19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19c500 con 0x7effcc106020 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19c670 con 0x7effcc100f50 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd2cec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc19c7e0 con 0x7effcc19bb10 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38900/0 (socket says 172.21.15.33:38900) 2024-01-25T16:11:22.988 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 -- 172.21.15.33:0/988907465 learned_addr learned my addr 172.21.15.33:0/988907465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd0a88640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 0x7effcc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd1289640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc19bb10 0x7effcc19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 -- 172.21.15.33:0/988907465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc19bb10 msgr2=0x7effcc19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc19bb10 0x7effcc19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 -- 172.21.15.33:0/988907465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 msgr2=0x7effcc1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 0x7effcc1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effcbfff640 1 -- 172.21.15.33:0/988907465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb4009560 con 0x7effcc106020 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.983+0000 7effd0a88640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 0x7effcc1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effcbfff640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effbc00eb90 tx=0x7effbc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effc9ffb640 1 -- 172.21.15.33:0/988907465 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0099b0 con 0x7effcc106020 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effc9ffb640 1 -- 172.21.15.33:0/988907465 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effbc00ee90 con 0x7effcc106020 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc1a3dc0 con 0x7effcc106020 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effc9ffb640 1 -- 172.21.15.33:0/988907465 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc018a60 con 0x7effcc106020 2024-01-25T16:11:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.984+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1a4210 con 0x7effcc106020 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effc9ffb640 1 -- 172.21.15.33:0/988907465 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7effbc018bc0 con 0x7effcc106020 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 Processor -- start 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 -- start start 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 0x55ed94dfcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94d60070 con 0x55ed94df19b0 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94d5d760 con 0x55ed94dfa780 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.987+0000 7effd5984740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94d59050 con 0x55ed94df1090 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38904/0 (socket says 172.21.15.33:38904) 2024-01-25T16:11:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 -- 172.21.15.33:0/416838711 learned_addr learned my addr 172.21.15.33:0/416838711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effc9ffb640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 0x7effb80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd1289640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effcbfff640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 0x55ed94dfcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd1289640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 0x7effb80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 -- 172.21.15.33:0/416838711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 msgr2=0x55ed94df1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 -- 172.21.15.33:0/416838711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 msgr2=0x55ed94dfcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 0x55ed94dfcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 -- 172.21.15.33:0/416838711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed94d50e70 con 0x55ed94df19b0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effcbfff640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 0x55ed94dfcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effc9ffb640 1 -- 172.21.15.33:0/988907465 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7effbc014070 con 0x7effcc106020 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd0a88640 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7effb4009930 tx=0x7effb4004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9b0484d621bc4b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effc2ffd640 1 -- 172.21.15.33:0/416838711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4004a70 con 0x55ed94df19b0 2024-01-25T16:11:22.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effd1289640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 0x7effb80a39b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effc400bea0 tx=0x7effc400b2e0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effc2ffd640 1 -- 172.21.15.33:0/416838711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb4004040 con 0x55ed94df19b0 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.988+0000 7effc2ffd640 1 -- 172.21.15.33:0/416838711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb4005390 con 0x55ed94df19b0 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 -- 172.21.15.33:0/416838711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 msgr2=0x55ed94dfa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7effb4009930 tx=0x7effb4004e80 comp rx=0 tx=0).stop 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 -- 172.21.15.33:0/416838711 shutdown_connections 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94dfa780 0x55ed94dfcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94dfa240 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 --2- 172.21.15.33:0/416838711 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 -- 172.21.15.33:0/416838711 >> 172.21.15.33:0/416838711 conn(0x55ed94deb9b0 msgr2=0x55ed94debd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 -- 172.21.15.33:0/416838711 shutdown_connections 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.989+0000 7effd5984740 1 -- 172.21.15.33:0/416838711 wait complete. 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 -- start start 2024-01-25T16:11:22.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 Processor -- start 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94df76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94df8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94d121e0 con 0x55ed94df19b0 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94d11e80 con 0x55ed94df1090 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94cd70c0 con 0x55ed94dfa780 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd5984740 10 client.0 _ll_register_callbacks cb 0x55ed94cfcac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38914/0 (socket says 172.21.15.33:38914) 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effcbfff640 1 -- 172.21.15.33:0/2088350796 learned_addr learned my addr 172.21.15.33:0/2088350796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.990+0000 7effd0a88640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94df76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effd1289640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94df8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 -- 172.21.15.33:0/2088350796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 msgr2=0x55ed94df8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94df8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 -- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 msgr2=0x55ed94df76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94df76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 -- 172.21.15.33:0/2088350796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc10e400 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effd0a88640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94df76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effcbfff640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effbc00cba0 tx=0x7effbc0699f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4298bfeeea5fd159 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effabfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc0a2680 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7effbc0caec0 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed94d52040 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7effbc0cb080 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effd5984740 10 client.15684 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ed94d576d0 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.991+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effbc069020 con 0x55ed94df19b0 2024-01-25T16:11:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7effbc0a21f0 con 0x55ed94df19b0 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 10 client.15684 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 10 client.15684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 20 client.15684 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e0a440 0x55ed94e0c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 4 client.15684 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed94d51d60 con 0x55ed94e0a440 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd0a88640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e0a440 0x55ed94e0c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effabfff640 10 client.15684 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.992+0000 7effd0a88640 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e0a440 0x55ed94e0c870 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7bb8aad942fa84e server_cookie=1c5f5342fb29b91f in_seq=0 out_seq=0 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effabfff640 1 -- 172.21.15.33:0/2088350796 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ed94d51d60 con 0x55ed94e0a440 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effabfff640 10 client.15684 handle_command_reply: tid=0 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 client.15684 shutdown 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 msgr2=0x55ed94df7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7effbc00cba0 tx=0x7effbc0699f0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 shutdown_connections 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e0a440 0x55ed94e0c870 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94df8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94df7bf0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effabfff640 0 client.15684 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 --2- 172.21.15.33:0/2088350796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94df76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.993+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 >> 172.21.15.33:0/2088350796 conn(0x55ed94deb9b0 msgr2=0x55ed94ded700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.994+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 shutdown_connections 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.994+0000 7effd5984740 1 -- 172.21.15.33:0/2088350796 wait complete. 2024-01-25T16:11:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:22.994+0000 7effd5984740 20 client.15684 trim_cache size 0 max 16384 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 Processor -- start 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 -- start start 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df19b0 0x55ed94df7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94de7270 con 0x55ed94dfa780 2024-01-25T16:11:23.014 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94e11850 con 0x55ed94df19b0 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd5984740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94dec7b0 con 0x55ed94df1090 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38928/0 (socket says 172.21.15.33:38928) 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 -- 172.21.15.33:0/3867657008 learned_addr learned my addr 172.21.15.33:0/3867657008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd0a88640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effcbfff640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df19b0 0x55ed94df7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 -- 172.21.15.33:0/3867657008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 msgr2=0x55ed94df9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 -- 172.21.15.33:0/3867657008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df19b0 msgr2=0x55ed94df7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df19b0 0x55ed94df7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.007+0000 7effd1289640 1 -- 172.21.15.33:0/3867657008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed94cf4060 con 0x55ed94dfa780 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd0a88640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd1289640 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7effc400bad0 tx=0x7effc400b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbe3ff807be1d54 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:23.015 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effc27fc640 1 -- 172.21.15.33:0/3867657008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4018070 con 0x55ed94dfa780 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effc27fc640 1 -- 172.21.15.33:0/3867657008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effc4024070 con 0x55ed94dfa780 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effc27fc640 1 -- 172.21.15.33:0/3867657008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4028650 con 0x55ed94dfa780 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 -- 172.21.15.33:0/3867657008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 msgr2=0x55ed94df6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7effc400bad0 tx=0x7effc400b600 comp rx=0 tx=0).stop 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 -- 172.21.15.33:0/3867657008 shutdown_connections 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94dfa780 0x55ed94df6690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df19b0 0x55ed94df7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 --2- 172.21.15.33:0/3867657008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94df1090 0x55ed94df9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.008+0000 7effd5984740 1 -- 172.21.15.33:0/3867657008 >> 172.21.15.33:0/3867657008 conn(0x55ed94deb9b0 msgr2=0x55ed94df5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- 172.21.15.33:0/3867657008 shutdown_connections 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- 172.21.15.33:0/3867657008 wait complete. 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- start start 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 Processor -- start 2024-01-25T16:11:23.016 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94e00150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94e00bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94e60a90 con 0x55ed94df19b0 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94e60c00 con 0x55ed94df1090 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed94e60dc0 con 0x55ed94dfa780 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effd5984740 10 client.0 _ll_register_callbacks cb 0x7effb4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.009+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38932/0 (socket says 172.21.15.33:38932) 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 -- 172.21.15.33:0/3012709624 learned_addr learned my addr 172.21.15.33:0/3012709624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effd0a88640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94e00150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effd1289640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94e00bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 -- 172.21.15.33:0/3012709624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 msgr2=0x55ed94e00bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.017 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94e00bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 -- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 msgr2=0x55ed94e00150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94e00150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 -- 172.21.15.33:0/3012709624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed94d19a80 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effcbfff640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effbc096e10 tx=0x7effbc099890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3700bbdc4658def5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effabfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc099ee0 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7effbc0a2680 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effb4004040 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7effbc0966e0 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effd5984740 10 client.15690 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55ed94d003a0 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.010+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effb4004040 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7effbc0962d0 con 0x55ed94df19b0 2024-01-25T16:11:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 10 client.15690 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 20 client.15690 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e621a0 0x55ed94e645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 4 client.15690 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ed94a58600 con 0x55ed94e621a0 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd0a88640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e621a0 0x55ed94e645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effabfff640 10 client.15690 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.011+0000 7effd0a88640 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e621a0 0x55ed94e645c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1035ed1a48203f1 server_cookie=836fa33f1ed0bf93 in_seq=0 out_seq=0 2024-01-25T16:11:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effabfff640 1 -- 172.21.15.33:0/3012709624 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ed94a58600 con 0x55ed94e621a0 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effabfff640 10 client.15690 handle_command_reply: tid=0 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 client.15690 shutdown 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 msgr2=0x55ed94e00690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7effbc096e10 tx=0x7effbc099890 comp rx=0 tx=0).stop 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 shutdown_connections 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ed94e621a0 0x55ed94e645c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ed94dfa780 0x55ed94e00bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ed94df19b0 0x55ed94e00690 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effabfff640 0 client.15690 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 --2- 172.21.15.33:0/3012709624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ed94df1090 0x55ed94e00150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 >> 172.21.15.33:0/3012709624 conn(0x55ed94deb9b0 msgr2=0x55ed94dea1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.068+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 shutdown_connections 2024-01-25T16:11:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.069+0000 7effd5984740 1 -- 172.21.15.33:0/3012709624 wait complete. 2024-01-25T16:11:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.069+0000 7effd5984740 20 client.15690 trim_cache size 0 max 16384 2024-01-25T16:11:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:11:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:11:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:11:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 msgr2=0x7effb80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 0x7effb80a39b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7effc400bea0 tx=0x7effc400b2e0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 msgr2=0x7effcc1977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7effbc00eb90 tx=0x7effbc00c6a0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 shutdown_connections 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7effb80a14f0 0x7effb80a39b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7effcc19bb10 0x7effcc19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effcc106020 0x7effcc1977e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 --2- 172.21.15.33:0/988907465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7effcc100f50 0x7effcc1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 >> 172.21.15.33:0/988907465 conn(0x7effcc0fc7b0 msgr2=0x7effcc0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 shutdown_connections 2024-01-25T16:11:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:23.070+0000 7effd2cec640 1 -- 172.21.15.33:0/988907465 wait complete. 2024-01-25T16:11:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:23 smithi033 ceph-mon[31417]: pgmap v109: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-01-25T16:11:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:23 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:23 smithi152 ceph-mon[36015]: pgmap v109: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-01-25T16:11:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:23 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:23 smithi076 ceph-mon[35642]: pgmap v109: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-01-25T16:11:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:23 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:11:23.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 69809 2.87 MB/sec execute 20 sec latency 1812.093 ms 2024-01-25T16:11:24.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 69809 2.74 MB/sec execute 21 sec latency 2812.221 ms 2024-01-25T16:11:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:25 smithi033 ceph-mon[31417]: pgmap v110: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 877 KiB/s wr, 71 op/s 2024-01-25T16:11:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:25 smithi152 ceph-mon[36015]: pgmap v110: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 877 KiB/s wr, 71 op/s 2024-01-25T16:11:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:25 smithi076 ceph-mon[35642]: pgmap v110: 97 pgs: 97 active+clean; 37 MiB data, 584 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 877 KiB/s wr, 71 op/s 2024-01-25T16:11:25.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 70606 2.90 MB/sec execute 22 sec latency 3479.399 ms 2024-01-25T16:11:26.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 72572 3.25 MB/sec execute 23 sec latency 9.526 ms 2024-01-25T16:11:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:27 smithi033 ceph-mon[31417]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-01-25T16:11:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:27 smithi152 ceph-mon[36015]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-01-25T16:11:27.671 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:27 smithi076 ceph-mon[35642]: pgmap v111: 97 pgs: 97 active+clean; 35 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-01-25T16:11:27.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73386 3.19 MB/sec execute 24 sec latency 831.191 ms 2024-01-25T16:11:28.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73386 3.07 MB/sec execute 25 sec latency 1831.257 ms 2024-01-25T16:11:29.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:29 smithi076 ceph-mon[35642]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-01-25T16:11:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:29 smithi033 ceph-mon[31417]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-01-25T16:11:29.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:29 smithi152 ceph-mon[36015]: pgmap v112: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-01-25T16:11:29.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73386 2.95 MB/sec execute 26 sec latency 2831.361 ms 2024-01-25T16:11:30.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73426 2.85 MB/sec execute 27 sec latency 3512.006 ms 2024-01-25T16:11:31.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:31 smithi033 ceph-mon[31417]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 104 op/s 2024-01-25T16:11:31.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:31 smithi152 ceph-mon[36015]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 104 op/s 2024-01-25T16:11:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:31 smithi076 ceph-mon[35642]: pgmap v113: 97 pgs: 97 active+clean; 35 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 764 KiB/s wr, 104 op/s 2024-01-25T16:11:31.764 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73426 2.74 MB/sec execute 28 sec latency 1279.330 ms 2024-01-25T16:11:32.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73426 2.65 MB/sec execute 29 sec latency 2279.396 ms 2024-01-25T16:11:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:33 smithi033 ceph-mon[31417]: pgmap v114: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-01-25T16:11:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:33 smithi152 ceph-mon[36015]: pgmap v114: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-01-25T16:11:33.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:33 smithi076 ceph-mon[35642]: pgmap v114: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 133 op/s 2024-01-25T16:11:33.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73426 2.56 MB/sec execute 30 sec latency 3279.482 ms 2024-01-25T16:11:34.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73426 2.48 MB/sec execute 31 sec latency 4279.566 ms 2024-01-25T16:11:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:35 smithi033 ceph-mon[31417]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 666 KiB/s wr, 92 op/s 2024-01-25T16:11:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:35 smithi152 ceph-mon[36015]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 666 KiB/s wr, 92 op/s 2024-01-25T16:11:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:35 smithi076 ceph-mon[35642]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 666 KiB/s wr, 92 op/s 2024-01-25T16:11:35.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 74290 2.63 MB/sec execute 32 sec latency 4954.995 ms 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 -- 172.21.15.33:0/3371706493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0106000 msgr2=0x7f79a010dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 --2- 172.21.15.33:0/3371706493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0106000 0x7f79a010dbd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7998009cd0 tx=0x7f799802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 -- 172.21.15.33:0/3371706493 shutdown_connections 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 --2- 172.21.15.33:0/3371706493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0106000 0x7f79a010dbd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 --2- 172.21.15.33:0/3371706493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a01018e0 0x7f79a0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 --2- 172.21.15.33:0/3371706493 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0100f30 0x7f79a0101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 -- 172.21.15.33:0/3371706493 >> 172.21.15.33:0/3371706493 conn(0x7f79a00fc7d0 msgr2=0x7f79a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 -- 172.21.15.33:0/3371706493 shutdown_connections 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 -- 172.21.15.33:0/3371706493 wait complete. 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.162+0000 7f79a719e640 1 Processor -- start 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 -- start start 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 0x7f79a0073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a0073a70 0x7f79a01a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01a59a0 con 0x7f79a0100f30 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01a5b10 con 0x7f79a0073a70 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a719e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a01a5cd0 con 0x7f79a0106000 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60440/0 (socket says 172.21.15.33:60440) 2024-01-25T16:11:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1093529875 learned_addr learned my addr 172.21.15.33:0/1093529875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f799ffff640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 0x7f79a0073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a573b640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a0073a70 0x7f79a01a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1093529875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 msgr2=0x7f79a0073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 0x7f79a0073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1093529875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a0073a70 msgr2=0x7f79a01a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a0073a70 0x7f79a01a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1093529875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998009900 con 0x7f79a0100f30 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f799ffff640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 0x7f79a0073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.163+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f799400e9f0 tx=0x7f799400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.164+0000 7f799dffb640 1 -- 172.21.15.33:0/1093529875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994009720 con 0x7f79a0100f30 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.164+0000 7f799dffb640 1 -- 172.21.15.33:0/1093529875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f799400cd80 con 0x7f79a0100f30 2024-01-25T16:11:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.164+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a01a5fb0 con 0x7f79a0100f30 2024-01-25T16:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.164+0000 7f799dffb640 1 -- 172.21.15.33:0/1093529875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7994010430 con 0x7f79a0100f30 2024-01-25T16:11:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.164+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01a63e0 con 0x7f79a0100f30 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.166+0000 7f799dffb640 1 -- 172.21.15.33:0/1093529875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7994005020 con 0x7f79a0100f30 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f799dffb640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 0x7f79900a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 Processor -- start 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f799dffb640 1 -- 172.21.15.33:0/1093529875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7994014070 con 0x7f79a0100f30 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f799ffff640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 0x7f79900a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 -- start start 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d29b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c8d070 con 0x565241d1e090 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c8a760 con 0x565241d27780 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a99fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c86050 con 0x565241d1e9b0 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60456/0 (socket says 172.21.15.33:60456) 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1990992589 learned_addr learned my addr 172.21.15.33:0/1990992589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f799ffff640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 0x7f79900a39e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a0074a80 tx=0x7f798c005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a573b640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d29b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f799ffff640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1990992589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 msgr2=0x565241d27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d27240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1990992589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 msgr2=0x565241d29b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.167+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d29b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f79a4f3a640 1 -- 172.21.15.33:0/1990992589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565241c7de70 con 0x565241d1e090 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f799ffff640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7994009380 tx=0x7f799409a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d757e76e3ae35bd server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f798affd640 1 -- 172.21.15.33:0/1990992589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799409e190 con 0x565241d1e090 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f798affd640 1 -- 172.21.15.33:0/1990992589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79940a23d0 con 0x565241d1e090 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f798affd640 1 -- 172.21.15.33:0/1990992589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940054d0 con 0x565241d1e090 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f79a99fd740 1 -- 172.21.15.33:0/1990992589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 msgr2=0x565241d1e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.168+0000 7f79a99fd740 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7994009380 tx=0x7f799409a880 comp rx=0 tx=0).stop 2024-01-25T16:11:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- 172.21.15.33:0/1990992589 shutdown_connections 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d29b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- 172.21.15.33:0/1990992589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d1e470 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- 172.21.15.33:0/1990992589 >> 172.21.15.33:0/1990992589 conn(0x565241d189b0 msgr2=0x565241d18d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- 172.21.15.33:0/1990992589 shutdown_connections 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- 172.21.15.33:0/1990992589 wait complete. 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- start start 2024-01-25T16:11:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 Processor -- start 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d1e9b0 0x565241d24c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d27780 0x565241d251b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c3f1e0 con 0x565241d1e090 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c3ee80 con 0x565241d1e9b0 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241c040c0 con 0x565241d27780 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.169+0000 7f79a99fd740 10 client.0 _ll_register_callbacks cb 0x565241c29ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60468/0 (socket says 172.21.15.33:60468) 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 -- 172.21.15.33:0/2931720070 learned_addr learned my addr 172.21.15.33:0/2931720070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a573b640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d27780 0x565241d251b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f799ffff640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d1e9b0 0x565241d24c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 -- 172.21.15.33:0/2931720070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d27780 msgr2=0x565241d251b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d27780 0x565241d251b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 -- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d1e9b0 msgr2=0x565241d24c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d1e9b0 0x565241d24c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 -- 172.21.15.33:0/2931720070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a01a63e0 con 0x565241d1e090 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79940a1ea0 tx=0x7f799400cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77169d05fee53822 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f797ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940054e0 con 0x565241d1e090 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7994098180 con 0x565241d1e090 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565241c7f040 con 0x565241d1e090 2024-01-25T16:11:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7994099530 con 0x565241d1e090 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a99fd740 10 client.15702 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565241c846d0 con 0x565241d1e090 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.170+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7994088020 con 0x565241d1e090 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f799401f060 con 0x565241d1e090 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 10 client.15702 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 10 client.15702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 20 client.15702 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d39720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 4 client.15702 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565241c7ed60 con 0x565241d37300 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f799ffff640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d39720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f797ffff640 10 client.15702 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.171+0000 7f799ffff640 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d39720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e4ad88cae7330e5 server_cookie=3c264d2afc90106d in_seq=0 out_seq=0 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f797ffff640 1 -- 172.21.15.33:0/2931720070 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x565241c7ed60 con 0x565241d37300 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f797ffff640 10 client.15702 handle_command_reply: tid=0 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 client.15702 shutdown 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 msgr2=0x565241d24730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f79940a1ea0 tx=0x7f799400cfc0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 shutdown_connections 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d39720 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d27780 0x565241d251b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d1e9b0 0x565241d24c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f797ffff640 0 client.15702 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 --2- 172.21.15.33:0/2931720070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d24730 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 >> 172.21.15.33:0/2931720070 conn(0x565241d189b0 msgr2=0x565241d1a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 shutdown_connections 2024-01-25T16:11:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 1 -- 172.21.15.33:0/2931720070 wait complete. 2024-01-25T16:11:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.172+0000 7f79a99fd740 20 client.15702 trim_cache size 0 max 16384 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.185+0000 7f79a99fd740 1 Processor -- start 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.185+0000 7f79a99fd740 1 -- start start 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.185+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d24fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d21990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a99fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d19a30 con 0x565241d1e090 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a99fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d20380 con 0x565241d27780 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a99fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d22cb0 con 0x565241d1e9b0 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60480/0 (socket says 172.21.15.33:60480) 2024-01-25T16:11:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 -- 172.21.15.33:0/76097316 learned_addr learned my addr 172.21.15.33:0/76097316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a573b640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d21990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f799ffff640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d24fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 -- 172.21.15.33:0/76097316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 msgr2=0x565241d24fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d24fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 -- 172.21.15.33:0/76097316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 msgr2=0x565241d21990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d21990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 -- 172.21.15.33:0/76097316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565241c21060 con 0x565241d1e090 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a573b640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d21990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7994009380 tx=0x7f79940caa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5867e42f37cda583 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f798a7fc640 1 -- 172.21.15.33:0/76097316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940054e0 con 0x565241d1e090 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f798a7fc640 1 -- 172.21.15.33:0/76097316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7994088070 con 0x565241d1e090 2024-01-25T16:11:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.186+0000 7f798a7fc640 1 -- 172.21.15.33:0/76097316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799409f0e0 con 0x565241d1e090 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- 172.21.15.33:0/76097316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 msgr2=0x565241d26e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7994009380 tx=0x7f79940caa40 comp rx=0 tx=0).stop 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- 172.21.15.33:0/76097316 shutdown_connections 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d21990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d24fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 --2- 172.21.15.33:0/76097316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d26e60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- 172.21.15.33:0/76097316 >> 172.21.15.33:0/76097316 conn(0x565241d189b0 msgr2=0x565241d23000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- 172.21.15.33:0/76097316 shutdown_connections 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- 172.21.15.33:0/76097316 wait complete. 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 -- start start 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.187+0000 7f79a99fd740 1 Processor -- start 2024-01-25T16:11:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d2d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d2db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d8d7d0 con 0x565241d1e090 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d8d910 con 0x565241d27780 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x565241d8dad0 con 0x565241d1e9b0 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 10 client.0 _ll_register_callbacks cb 0x7f7994096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60486/0 (socket says 172.21.15.33:60486) 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 -- 172.21.15.33:0/775369470 learned_addr learned my addr 172.21.15.33:0/775369470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f799ffff640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d2d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a573b640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d2db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 -- 172.21.15.33:0/775369470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 msgr2=0x565241d2d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d2d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 -- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 msgr2=0x565241d2db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d2db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 -- 172.21.15.33:0/775369470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565241c846d0 con 0x565241d1e090 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a4f3a640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f799400c6a0 tx=0x7f7994098fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e000e389b4b4da27 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f797ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79940054e0 con 0x565241d1e090 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f7994089080 con 0x565241d1e090 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565241d140b0 con 0x565241d1e090 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79940999a0 con 0x565241d1e090 2024-01-25T16:11:36.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 10 client.15708 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.188+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x565241c46a80 con 0x565241d1e090 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565241d140b0 con 0x565241d1e090 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f799401f070 con 0x565241d1e090 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 10 client.15708 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 20 client.15708 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d90de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 4 client.15708 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f79940a23d0 con 0x565241d37300 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f799ffff640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d90de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f799ffff640 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d90de0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d21b642fc032ede2 server_cookie=503804265a523f83 in_seq=0 out_seq=0 2024-01-25T16:11:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.189+0000 7f797ffff640 10 client.15708 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f797ffff640 1 -- 172.21.15.33:0/775369470 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f79940a23d0 con 0x565241d37300 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f797ffff640 10 client.15708 handle_command_reply: tid=0 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f79a99fd740 1 client.15708 shutdown 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 msgr2=0x565241d2d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f799400c6a0 tx=0x7f7994098fa0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 shutdown_connections 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.237+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x565241d37300 0x565241d90de0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x565241d27780 0x565241d2db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x565241d1e9b0 0x565241d2d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 --2- 172.21.15.33:0/775369470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565241d1e090 0x565241d2d0f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f797ffff640 0 client.15708 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 >> 172.21.15.33:0/775369470 conn(0x565241d189b0 msgr2=0x565241d171c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 shutdown_connections 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 1 -- 172.21.15.33:0/775369470 wait complete. 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a99fd740 20 client.15708 trim_cache size 0 max 16384 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:11:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:11:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 msgr2=0x7f79900a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 0x7f79900a39e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a0074a80 tx=0x7f798c005f90 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 msgr2=0x7f79a0072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f799400e9f0 tx=0x7f799400edf0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 shutdown_connections 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79900a1520 0x7f79900a39e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a0073a70 0x7f79a01a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a0106000 0x7f79a0073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 --2- 172.21.15.33:0/1093529875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a0100f30 0x7f79a0072ff0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.238+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 >> 172.21.15.33:0/1093529875 conn(0x7f79a00fc7d0 msgr2=0x7f79a00feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.239+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 shutdown_connections 2024-01-25T16:11:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:36.239+0000 7f79a719e640 1 -- 172.21.15.33:0/1093529875 wait complete. 2024-01-25T16:11:36.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:36 smithi033 ceph-mon[31417]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 KiB/s wr, 114 op/s 2024-01-25T16:11:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:36 smithi152 ceph-mon[36015]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 KiB/s wr, 114 op/s 2024-01-25T16:11:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:36 smithi076 ceph-mon[35642]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 KiB/s wr, 114 op/s 2024-01-25T16:11:36.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76319 2.85 MB/sec execute 33 sec latency 19.846 ms 2024-01-25T16:11:37.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76879 2.81 MB/sec execute 34 sec latency 922.888 ms 2024-01-25T16:11:38.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76879 2.73 MB/sec execute 35 sec latency 1923.009 ms 2024-01-25T16:11:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:39 smithi033 ceph-mon[31417]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 520 KiB/s wr, 54 op/s 2024-01-25T16:11:39.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:39 smithi152 ceph-mon[36015]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 520 KiB/s wr, 54 op/s 2024-01-25T16:11:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:39 smithi076 ceph-mon[35642]: pgmap v117: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 520 KiB/s wr, 54 op/s 2024-01-25T16:11:39.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76879 2.66 MB/sec execute 36 sec latency 2923.126 ms 2024-01-25T16:11:40.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76973 2.59 MB/sec execute 37 sec latency 3602.300 ms 2024-01-25T16:11:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:41 smithi033 ceph-mon[31417]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 508 KiB/s wr, 51 op/s 2024-01-25T16:11:41.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:41 smithi152 ceph-mon[36015]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 508 KiB/s wr, 51 op/s 2024-01-25T16:11:41.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:41 smithi076 ceph-mon[35642]: pgmap v118: 97 pgs: 97 active+clean; 37 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 508 KiB/s wr, 51 op/s 2024-01-25T16:11:41.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76973 2.52 MB/sec execute 38 sec latency 1238.366 ms 2024-01-25T16:11:42.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76973 2.46 MB/sec execute 39 sec latency 2238.479 ms 2024-01-25T16:11:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:43 smithi033 ceph-mon[31417]: pgmap v119: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 735 KiB/s wr, 105 op/s 2024-01-25T16:11:43.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:43 smithi152 ceph-mon[36015]: pgmap v119: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 735 KiB/s wr, 105 op/s 2024-01-25T16:11:43.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:43 smithi076 ceph-mon[35642]: pgmap v119: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 735 KiB/s wr, 105 op/s 2024-01-25T16:11:43.765 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76973 2.40 MB/sec execute 40 sec latency 3238.572 ms 2024-01-25T16:11:44.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76973 2.34 MB/sec execute 41 sec latency 4238.676 ms 2024-01-25T16:11:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:45 smithi033 ceph-mon[31417]: pgmap v120: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 364 KiB/s wr, 76 op/s 2024-01-25T16:11:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:45 smithi152 ceph-mon[36015]: pgmap v120: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 364 KiB/s wr, 76 op/s 2024-01-25T16:11:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:45 smithi076 ceph-mon[35642]: pgmap v120: 97 pgs: 97 active+clean; 34 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 364 KiB/s wr, 76 op/s 2024-01-25T16:11:45.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77034 2.30 MB/sec execute 42 sec latency 4913.877 ms 2024-01-25T16:11:46.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:46 smithi033 ceph-mon[31417]: pgmap v121: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-01-25T16:11:46.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:46 smithi152 ceph-mon[36015]: pgmap v121: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-01-25T16:11:46.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:46 smithi076 ceph-mon[35642]: pgmap v121: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 657 KiB/s wr, 90 op/s 2024-01-25T16:11:46.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77034 2.25 MB/sec execute 43 sec latency 1257.056 ms 2024-01-25T16:11:47.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77034 2.19 MB/sec execute 44 sec latency 2257.163 ms 2024-01-25T16:11:48.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77034 2.15 MB/sec execute 45 sec latency 3257.262 ms 2024-01-25T16:11:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 -- 172.21.15.33:0/1896484134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca8104b70 msgr2=0x7fdca8104f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 --2- 172.21.15.33:0/1896484134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca8104b70 0x7fdca8104f50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009900 tx=0x7fdc9802bc30 comp rx=0 tx=0).stop 2024-01-25T16:11:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 -- 172.21.15.33:0/1896484134 shutdown_connections 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 --2- 172.21.15.33:0/1896484134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca81007e0 0x7fdca810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 --2- 172.21.15.33:0/1896484134 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca80ffe40 0x7fdca81002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 --2- 172.21.15.33:0/1896484134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca8104b70 0x7fdca8104f50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 -- 172.21.15.33:0/1896484134 >> 172.21.15.33:0/1896484134 conn(0x7fdca80769c0 msgr2=0x7fdca8076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 -- 172.21.15.33:0/1896484134 shutdown_connections 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.336+0000 7fdcae007640 1 -- 172.21.15.33:0/1896484134 wait complete. 2024-01-25T16:11:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 Processor -- start 2024-01-25T16:11:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 -- start start 2024-01-25T16:11:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca81007e0 0x7fdca8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 0x7fdca81a3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8198d50 con 0x7fdca80ffe40 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81a3580 con 0x7fdca8104b70 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.337+0000 7fdcae007640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81a36c0 con 0x7fdca81007e0 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53262/0 (socket says 172.21.15.33:53262) 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 -- 172.21.15.33:0/2462740374 learned_addr learned my addr 172.21.15.33:0/2462740374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca81007e0 0x7fdca8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 0x7fdca81a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 -- 172.21.15.33:0/2462740374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca81007e0 msgr2=0x7fdca8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca81007e0 0x7fdca8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 -- 172.21.15.33:0/2462740374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 msgr2=0x7fdca81a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 0x7fdca81a3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 -- 172.21.15.33:0/2462740374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009560 con 0x7fdca80ffe40 2024-01-25T16:11:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 0x7fdca81a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdc98036040 tx=0x7fdc98004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca4ff9640 1 -- 172.21.15.33:0/2462740374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980042d0 con 0x7fdca80ffe40 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca4ff9640 1 -- 172.21.15.33:0/2462740374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc98004430 con 0x7fdca80ffe40 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca81a38c0 con 0x7fdca80ffe40 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdca4ff9640 1 -- 172.21.15.33:0/2462740374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98003cc0 con 0x7fdca80ffe40 2024-01-25T16:11:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.338+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca81a3dd0 con 0x7fdca80ffe40 2024-01-25T16:11:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.341+0000 7fdca4ff9640 1 -- 172.21.15.33:0/2462740374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fdc98005000 con 0x7fdca80ffe40 2024-01-25T16:11:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 Processor -- start 2024-01-25T16:11:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 -- start start 2024-01-25T16:11:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca4ff9640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 0x7fdc900a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f02e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575e72070 con 0x558575f10b90 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575e6f760 con 0x558575f02a50 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575e6b050 con 0x558575f03370 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca4ff9640 1 -- 172.21.15.33:0/2462740374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc980ef2b0 con 0x7fdca80ffe40 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 0x7fdc900a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53278/0 (socket says 172.21.15.33:53278) 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 -- 172.21.15.33:0/794010701 learned_addr learned my addr 172.21.15.33:0/794010701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca77fe640 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f02e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 -- 172.21.15.33:0/794010701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 msgr2=0x558575f10650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f10650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 -- 172.21.15.33:0/794010701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 msgr2=0x558575f02e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f02e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.342+0000 7fdca7fff640 1 -- 172.21.15.33:0/794010701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558575e62e70 con 0x558575f10b90 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.343+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 0x7fdc900a39b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c40 tx=0x7fdca0009d70 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.343+0000 7fdca7fff640 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdca0011ce0 tx=0x7fdca001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2a0c4c86756411 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.343+0000 7fdc8dffb640 1 -- 172.21.15.33:0/794010701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00144f0 con 0x558575f10b90 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.343+0000 7fdc8dffb640 1 -- 172.21.15.33:0/794010701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca001a040 con 0x558575f10b90 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.343+0000 7fdc8dffb640 1 -- 172.21.15.33:0/794010701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca00052b0 con 0x558575f10b90 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/794010701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 msgr2=0x558575f12f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdca0011ce0 tx=0x7fdca001b000 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/794010701 shutdown_connections 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f10b90 0x558575f12f80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f10650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/794010701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f02e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/794010701 >> 172.21.15.33:0/794010701 conn(0x558575efd9b0 msgr2=0x558575efdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/794010701 shutdown_connections 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/794010701 wait complete. 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- start start 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 Processor -- start 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f0db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 0x558575f0e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575e241e0 con 0x558575f03370 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575e23e80 con 0x558575f02a50 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.344+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575de90c0 con 0x558575f10b90 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdcb0bc6740 10 client.0 _ll_register_callbacks cb 0x558575e0eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53294/0 (socket says 172.21.15.33:53294) 2024-01-25T16:11:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 -- 172.21.15.33:0/1669416390 learned_addr learned my addr 172.21.15.33:0/1669416390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca77fe640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f0db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca7fff640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 0x558575f0e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 -- 172.21.15.33:0/1669416390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 msgr2=0x558575f0e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 0x558575f0e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 -- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 msgr2=0x558575f0db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f0db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 -- 172.21.15.33:0/1669416390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca0011aa0 con 0x558575f03370 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca7fff640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 0x558575f0e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00cef0 tx=0x7fdc9c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47aa74d3f10f9164 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdc86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0041d0 con 0x558575f03370 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc9c0119f0 con 0x558575f03370 2024-01-25T16:11:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558575e64040 con 0x558575f03370 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc9c011bb0 con 0x558575f03370 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdcb0bc6740 10 client.15720 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.345+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558575e696d0 con 0x558575f03370 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc9c026080 con 0x558575f03370 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc9c022410 con 0x558575f03370 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 10 client.15720 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 20 client.15720 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f20b20 0x558575f22f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 4 client.15720 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558575e63d60 con 0x558575f20b20 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.346+0000 7fdca77fe640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f20b20 0x558575f22f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.347+0000 7fdc86ffd640 10 client.15720 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.347+0000 7fdca77fe640 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f20b20 0x558575f22f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b4b76f381b167b9 server_cookie=7d75bdbf8426b4a in_seq=0 out_seq=0 2024-01-25T16:11:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.347+0000 7fdc86ffd640 1 -- 172.21.15.33:0/1669416390 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558575e63d60 con 0x558575f20b20 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.347+0000 7fdc86ffd640 10 client.15720 handle_command_reply: tid=0 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.347+0000 7fdcb0bc6740 1 client.15720 shutdown 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 msgr2=0x558575f0e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00cef0 tx=0x7fdc9c0039c0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 shutdown_connections 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f20b20 0x558575f22f40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f10b90 0x558575f0e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f0e090 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdc86ffd640 0 client.15720 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/1669416390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f02a50 0x558575f0db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 >> 172.21.15.33:0/1669416390 conn(0x558575efd9b0 msgr2=0x558575eff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 shutdown_connections 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/1669416390 wait complete. 2024-01-25T16:11:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.348+0000 7fdcb0bc6740 20 client.15720 trim_cache size 0 max 16384 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 Processor -- start 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 -- start start 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f0e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575efe9b0 con 0x558575f02a50 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575ef7f10 con 0x558575f10b90 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.361+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575f12f80 con 0x558575f03370 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53296/0 (socket says 172.21.15.33:53296) 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 -- 172.21.15.33:0/2001370377 learned_addr learned my addr 172.21.15.33:0/2001370377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f0e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 -- 172.21.15.33:0/2001370377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 msgr2=0x558575f0e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f0e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 -- 172.21.15.33:0/2001370377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 msgr2=0x558575f0adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f0adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 -- 172.21.15.33:0/2001370377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558575e06060 con 0x558575f02a50 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f0e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b8be0 tx=0x7fdc980b84b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e19f6775cec9e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdc8d7fa640 1 -- 172.21.15.33:0/2001370377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98003e20 con 0x558575f02a50 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdc8d7fa640 1 -- 172.21.15.33:0/2001370377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc980bc170 con 0x558575f02a50 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.362+0000 7fdc8d7fa640 1 -- 172.21.15.33:0/2001370377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980ba510 con 0x558575f02a50 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2001370377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 msgr2=0x558575f10280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b8be0 tx=0x7fdc980b84b0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2001370377 shutdown_connections 2024-01-25T16:11:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f03370 0x558575f0e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2001370377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f02a50 0x558575f10280 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2001370377 >> 172.21.15.33:0/2001370377 conn(0x558575efd9b0 msgr2=0x558575f0c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2001370377 shutdown_connections 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2001370377 wait complete. 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 -- start start 2024-01-25T16:11:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.363+0000 7fdcb0bc6740 1 Processor -- start 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 0x558575f17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575f16d80 con 0x558575f03370 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575f16ef0 con 0x558575f10b90 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558575f77030 con 0x558575f02a50 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdcb0bc6740 10 client.0 _ll_register_callbacks cb 0x7fdca0014670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53308/0 (socket says 172.21.15.33:53308) 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 -- 172.21.15.33:0/2123060093 learned_addr learned my addr 172.21.15.33:0/2123060093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca7fff640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 0x558575f17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 -- 172.21.15.33:0/2123060093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 msgr2=0x558575f17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 0x558575f17c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 -- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 msgr2=0x558575f166b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f166b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 -- 172.21.15.33:0/2123060093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca81a3dd0 con 0x558575f03370 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 0x558575f17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdca6ffd640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c000c00 tx=0x7fdc9c022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595ea337c2e23875 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdc86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.364+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c005240 con 0x558575f03370 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fdc9c0053a0 con 0x558575f03370 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdca001a040 con 0x558575f03370 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc9c005560 con 0x558575f03370 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 10 client.15726 fetch_fsmap learned FSMap version 26 2024-01-25T16:11:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558575e2ba80 con 0x558575f03370 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdca001a040 con 0x558575f03370 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc9c004970 con 0x558575f03370 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 10 client.15726 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 20 client.15726 populate_metadata read hostname 'smithi033' 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f78720 0x558575f7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 4 client.15726 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558575b6a600 con 0x558575f78720 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.365+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f78720 0x558575f7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.366+0000 7fdc86ffd640 10 client.15726 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:49.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.366+0000 7fdca77fe640 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f78720 0x558575f7ab40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a3a2e39baace3 server_cookie=596939f2311339e9 in_seq=0 out_seq=0 2024-01-25T16:11:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdc86ffd640 1 -- 172.21.15.33:0/2123060093 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558575b6a600 con 0x558575f78720 2024-01-25T16:11:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdc86ffd640 10 client.15726 handle_command_reply: tid=0 2024-01-25T16:11:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 client.15726 shutdown 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 msgr2=0x558575f16170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c000c00 tx=0x7fdc9c022c90 comp rx=0 tx=0).stop 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 shutdown_connections 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558575f78720 0x558575f7ab40 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558575f10b90 0x558575f166b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558575f03370 0x558575f16170 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdc86ffd640 0 client.15726 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 --2- 172.21.15.33:0/2123060093 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558575f02a50 0x558575f17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 >> 172.21.15.33:0/2123060093 conn(0x558575efd9b0 msgr2=0x558575efc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 shutdown_connections 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 1 -- 172.21.15.33:0/2123060093 wait complete. 2024-01-25T16:11:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.407+0000 7fdcb0bc6740 20 client.15726 trim_cache size 0 max 16384 2024-01-25T16:11:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:11:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:11:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:11:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 msgr2=0x7fdc900a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 0x7fdc900a39b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fdca0005c40 tx=0x7fdca0009d70 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 msgr2=0x7fdca8197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fdc98036040 tx=0x7fdc98004dd0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 shutdown_connections 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fdc900a14f0 0x7fdc900a39b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdca8104b70 0x7fdca81a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdca81007e0 0x7fdca8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 --2- 172.21.15.33:0/2462740374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca80ffe40 0x7fdca8197fc0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 >> 172.21.15.33:0/2462740374 conn(0x7fdca80769c0 msgr2=0x7fdca80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:11:49.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 shutdown_connections 2024-01-25T16:11:49.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:11:49.408+0000 7fdcae007640 1 -- 172.21.15.33:0/2462740374 wait complete. 2024-01-25T16:11:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:49 smithi033 ceph-mon[31417]: pgmap v122: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:49 smithi152 ceph-mon[36015]: pgmap v122: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:49 smithi076 ceph-mon[35642]: pgmap v122: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:49.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 77034 2.10 MB/sec execute 46 sec latency 4257.374 ms 2024-01-25T16:11:50.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 78087 2.26 MB/sec execute 47 sec latency 4925.237 ms 2024-01-25T16:11:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:51 smithi033 ceph-mon[31417]: pgmap v123: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:51.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:51 smithi152 ceph-mon[36015]: pgmap v123: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:51 smithi076 ceph-mon[35642]: pgmap v123: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 520 KiB/s wr, 68 op/s 2024-01-25T16:11:51.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80422 2.39 MB/sec execute 48 sec latency 52.293 ms 2024-01-25T16:11:52.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80422 2.34 MB/sec execute 49 sec latency 1052.396 ms 2024-01-25T16:11:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:53 smithi033 ceph-mon[31417]: pgmap v124: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 857 KiB/s wr, 108 op/s 2024-01-25T16:11:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:53 smithi152 ceph-mon[36015]: pgmap v124: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 857 KiB/s wr, 108 op/s 2024-01-25T16:11:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:53 smithi076 ceph-mon[35642]: pgmap v124: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 857 KiB/s wr, 108 op/s 2024-01-25T16:11:53.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80422 2.30 MB/sec execute 50 sec latency 2052.501 ms 2024-01-25T16:11:54.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80422 2.25 MB/sec execute 51 sec latency 3052.585 ms 2024-01-25T16:11:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:55 smithi033 ceph-mon[31417]: pgmap v125: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 629 KiB/s wr, 54 op/s 2024-01-25T16:11:55.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:55 smithi152 ceph-mon[36015]: pgmap v125: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 629 KiB/s wr, 54 op/s 2024-01-25T16:11:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:55 smithi076 ceph-mon[35642]: pgmap v125: 97 pgs: 97 active+clean; 34 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 629 KiB/s wr, 54 op/s 2024-01-25T16:11:55.766 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80476 2.21 MB/sec execute 52 sec latency 3726.606 ms 2024-01-25T16:11:56.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:56 smithi033 ceph-mon[31417]: pgmap v126: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1005 KiB/s wr, 80 op/s 2024-01-25T16:11:56.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:56 smithi152 ceph-mon[36015]: pgmap v126: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1005 KiB/s wr, 80 op/s 2024-01-25T16:11:56.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:56 smithi076 ceph-mon[35642]: pgmap v126: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 475 KiB/s rd, 1005 KiB/s wr, 80 op/s 2024-01-25T16:11:56.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80476 2.17 MB/sec execute 53 sec latency 1289.548 ms 2024-01-25T16:11:57.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80476 2.13 MB/sec execute 54 sec latency 2289.681 ms 2024-01-25T16:11:58.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80476 2.09 MB/sec execute 55 sec latency 3289.721 ms 2024-01-25T16:11:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:59 smithi033 ceph-mon[31417]: pgmap v127: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:11:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:11:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:11:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:11:59.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:11:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:11:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:59 smithi152 ceph-mon[36015]: pgmap v127: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:11:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:11:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:11:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:11:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:11:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:11:59.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:59 smithi076 ceph-mon[35642]: pgmap v127: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:11:59.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:11:59.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:11:59.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:11:59.669 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:11:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:11:59.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 80476 2.05 MB/sec execute 56 sec latency 4289.776 ms 2024-01-25T16:12:00.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 81280 2.13 MB/sec execute 57 sec latency 4960.418 ms 2024-01-25T16:12:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:01 smithi033 ceph-mon[31417]: pgmap v128: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:12:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:01 smithi152 ceph-mon[36015]: pgmap v128: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:12:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:01 smithi076 ceph-mon[35642]: pgmap v128: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 712 KiB/s wr, 66 op/s 2024-01-25T16:12:01.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 82549 2.20 MB/sec execute 58 sec latency 26.290 ms 2024-01-25T16:12:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 -- 172.21.15.33:0/905753907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 msgr2=0x7fe0481071b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 --2- 172.21.15.33:0/905753907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe0481071b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe038009900 tx=0x7fe03802bc30 comp rx=0 tx=0).stop 2024-01-25T16:12:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 -- 172.21.15.33:0/905753907 shutdown_connections 2024-01-25T16:12:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 --2- 172.21.15.33:0/905753907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 0x7fe04810db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 --2- 172.21.15.33:0/905753907 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 0x7fe0480ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 --2- 172.21.15.33:0/905753907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe0481071b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 -- 172.21.15.33:0/905753907 >> 172.21.15.33:0/905753907 conn(0x7fe048075d80 msgr2=0x7fe0480fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 -- 172.21.15.33:0/905753907 shutdown_connections 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.498+0000 7fe04e058640 1 -- 172.21.15.33:0/905753907 wait complete. 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 Processor -- start 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 -- start start 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 0x7fe048199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 0x7fe048199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04819e1f0 con 0x7fe048106dd0 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04819e360 con 0x7fe0480ff780 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe04e058640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04819e520 con 0x7fe0480fedc0 2024-01-25T16:12:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39258/0 (socket says 172.21.15.33:39258) 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.499+0000 7fe047fff640 1 -- 172.21.15.33:0/465863337 learned_addr learned my addr 172.21.15.33:0/465863337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe0477fe640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 0x7fe048199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe046ffd640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 0x7fe048199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 -- 172.21.15.33:0/465863337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 msgr2=0x7fe048199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 0x7fe048199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 -- 172.21.15.33:0/465863337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 msgr2=0x7fe048199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 0x7fe048199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 -- 172.21.15.33:0/465863337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038009560 con 0x7fe048106dd0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe047fff640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe04000eb10 tx=0x7fe04000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe044ff9640 1 -- 172.21.15.33:0/465863337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040004b20 con 0x7fe048106dd0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe044ff9640 1 -- 172.21.15.33:0/465863337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe040004c80 con 0x7fe048106dd0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04819e800 con 0x7fe048106dd0 2024-01-25T16:12:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe044ff9640 1 -- 172.21.15.33:0/465863337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040010690 con 0x7fe048106dd0 2024-01-25T16:12:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.500+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a6080 con 0x7fe048106dd0 2024-01-25T16:12:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.502+0000 7fe044ff9640 1 -- 172.21.15.33:0/465863337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe040005020 con 0x7fe048106dd0 2024-01-25T16:12:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 Processor -- start 2024-01-25T16:12:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe044ff9640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 0x7fe0300a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 -- start start 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe046ffd640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 0x7fe0300a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe044ff9640 1 -- 172.21.15.33:0/465863337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe040014070 con 0x7fe048106dd0 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 0x561300366470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x561300371b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613002d5070 con 0x5613003669b0 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613002d2760 con 0x56130036f780 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe050c1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613002ce050 con 0x561300366090 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39266/0 (socket says 172.21.15.33:39266) 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe046ffd640 1 -- 172.21.15.33:0/294810059 learned_addr learned my addr 172.21.15.33:0/294810059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x561300371b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe0477fe640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 0x561300366470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 -- 172.21.15.33:0/294810059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 msgr2=0x561300366470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 0x561300366470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 -- 172.21.15.33:0/294810059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 msgr2=0x56130036f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe046ffd640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 0x7fe0300a39b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009fd0 tx=0x7fe03c013820 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe047fff640 1 -- 172.21.15.33:0/294810059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613002c5e70 con 0x56130036f780 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.503+0000 7fe0477fe640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 0x561300366470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe047fff640 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x561300371b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0400cac30 tx=0x7fe040099ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4e02f8d014b1d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe02dffb640 1 -- 172.21.15.33:0/294810059 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04009a2e0 con 0x56130036f780 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe02dffb640 1 -- 172.21.15.33:0/294810059 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04009e430 con 0x56130036f780 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe02dffb640 1 -- 172.21.15.33:0/294810059 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400cb080 con 0x56130036f780 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 -- 172.21.15.33:0/294810059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 msgr2=0x561300371b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x561300371b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe0400cac30 tx=0x7fe040099ee0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 -- 172.21.15.33:0/294810059 shutdown_connections 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x561300371b70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 --2- 172.21.15.33:0/294810059 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561300366090 0x561300366470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.504+0000 7fe050c1c740 1 -- 172.21.15.33:0/294810059 >> 172.21.15.33:0/294810059 conn(0x5613003609b0 msgr2=0x561300360d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- 172.21.15.33:0/294810059 shutdown_connections 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- 172.21.15.33:0/294810059 wait complete. 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- start start 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 Processor -- start 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x56130036c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 0x56130036d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613002871e0 con 0x5613003669b0 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300286e80 con 0x561300366090 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130024c0c0 con 0x56130036f780 2024-01-25T16:12:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe050c1c740 10 client.0 _ll_register_callbacks cb 0x561300271ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe046ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39276/0 (socket says 172.21.15.33:39276) 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe046ffd640 1 -- 172.21.15.33:0/1849899192 learned_addr learned my addr 172.21.15.33:0/1849899192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.505+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x56130036c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe047fff640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 0x56130036d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 -- 172.21.15.33:0/1849899192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 msgr2=0x56130036d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 0x56130036d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 -- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 msgr2=0x56130036c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x56130036c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 -- 172.21.15.33:0/1849899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0481a6080 con 0x5613003669b0 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe047fff640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 0x56130036d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe046ffd640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00fc00 tx=0x7fe03c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cfda1fdeb6eb85 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c020b30 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe03c020c90 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613002c7040 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe03c020e50 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe050c1c740 10 client.15738 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5613002cc6d0 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.506+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe03c035020 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe03c0105e0 con 0x5613003669b0 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 10 client.15738 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 10 client.15738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 20 client.15738 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56130037f5a0 0x5613003819c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 4 client.15738 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613002c6d60 con 0x56130037f5a0 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56130037f5a0 0x5613003819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe026ffd640 10 client.15738 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.507+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56130037f5a0 0x5613003819c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267ffb071e73cfd0 server_cookie=1e791ee549d2bfc0 in_seq=0 out_seq=0 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe026ffd640 1 -- 172.21.15.33:0/1849899192 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5613002c6d60 con 0x56130037f5a0 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe026ffd640 10 client.15738 handle_command_reply: tid=0 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 client.15738 shutdown 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 msgr2=0x56130036ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00fc00 tx=0x7fe03c004730 comp rx=0 tx=0).stop 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 shutdown_connections 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56130037f5a0 0x5613003819c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56130036f780 0x56130036d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613003669b0 0x56130036ccb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe026ffd640 0 client.15738 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1849899192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x56130036c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 >> 172.21.15.33:0/1849899192 conn(0x5613003609b0 msgr2=0x561300362700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 shutdown_connections 2024-01-25T16:12:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 1 -- 172.21.15.33:0/1849899192 wait complete. 2024-01-25T16:12:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.508+0000 7fe050c1c740 20 client.15738 trim_cache size 0 max 16384 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 Processor -- start 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 -- start start 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x56130036d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x56130036b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56130035af30 con 0x561300366090 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300361b70 con 0x56130036f780 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.521+0000 7fe050c1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300386b70 con 0x5613003669b0 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39282/0 (socket says 172.21.15.33:39282) 2024-01-25T16:12:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 -- 172.21.15.33:0/1189926464 learned_addr learned my addr 172.21.15.33:0/1189926464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe046ffd640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x56130036d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe047fff640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x56130036b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 -- 172.21.15.33:0/1189926464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 msgr2=0x56130036d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x56130036d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 -- 172.21.15.33:0/1189926464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 msgr2=0x56130036b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x56130036b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 -- 172.21.15.33:0/1189926464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561300269060 con 0x561300366090 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe047fff640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x56130036b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe038009900 tx=0x7fe03802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5efcf6bae7e11a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe02d7fa640 1 -- 172.21.15.33:0/1189926464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038044040 con 0x561300366090 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe02d7fa640 1 -- 172.21.15.33:0/1189926464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe038046040 con 0x561300366090 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.522+0000 7fe02d7fa640 1 -- 172.21.15.33:0/1189926464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03804b690 con 0x561300366090 2024-01-25T16:12:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- 172.21.15.33:0/1189926464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 msgr2=0x56130036ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe038009900 tx=0x7fe03802c6c0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- 172.21.15.33:0/1189926464 shutdown_connections 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56130036f780 0x56130036b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x56130036d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1189926464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561300366090 0x56130036ee50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- 172.21.15.33:0/1189926464 >> 172.21.15.33:0/1189926464 conn(0x5613003609b0 msgr2=0x561300367ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- 172.21.15.33:0/1189926464 shutdown_connections 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- 172.21.15.33:0/1189926464 wait complete. 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 -- start start 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 Processor -- start 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x561300376850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.523+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x561300374cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300375900 con 0x56130036f780 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561300375a70 con 0x561300366090 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613003d5a70 con 0x5613003669b0 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 10 client.0 _ll_register_callbacks cb 0x7fe040005b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39290/0 (socket says 172.21.15.33:39290) 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 -- 172.21.15.33:0/1520216488 learned_addr learned my addr 172.21.15.33:0/1520216488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe046ffd640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x561300374cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x561300376850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 -- 172.21.15.33:0/1520216488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 msgr2=0x561300374cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x561300374cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 -- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 msgr2=0x561300376850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x561300376850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 -- 172.21.15.33:0/1520216488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04810e3f0 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe047fff640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe040004ff0 tx=0x7fe04000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee5fe58ffa73a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04009a2e0 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe0400b8030 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe04009e430 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0400a2670 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 10 client.15744 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.524+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56130028ea80 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe04009e430 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe04001f060 con 0x56130036f780 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 10 client.15744 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 20 client.15744 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5613003d6eb0 0x5613003d92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 4 client.15744 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:12:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5612fffcd600 con 0x5613003d6eb0 2024-01-25T16:12:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5613003d6eb0 0x5613003d92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe026ffd640 10 client.15744 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.525+0000 7fe0477fe640 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5613003d6eb0 0x5613003d92d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3339538d24550c4 server_cookie=62de4a70545a8208 in_seq=0 out_seq=0 2024-01-25T16:12:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:02 smithi033 ceph-mon[31417]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-01-25T16:12:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe026ffd640 1 -- 172.21.15.33:0/1520216488 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5612fffcd600 con 0x5613003d6eb0 2024-01-25T16:12:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe026ffd640 10 client.15744 handle_command_reply: tid=0 2024-01-25T16:12:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 client.15744 shutdown 2024-01-25T16:12:02.568 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 msgr2=0x561300375230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe040004ff0 tx=0x7fe04000efd0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 shutdown_connections 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5613003d6eb0 0x5613003d92d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56130036f780 0x561300375230 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5613003669b0 0x561300374cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 --2- 172.21.15.33:0/1520216488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561300366090 0x561300376850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 >> 172.21.15.33:0/1520216488 conn(0x5613003609b0 msgr2=0x56130036b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe026ffd640 0 client.15744 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 shutdown_connections 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.565+0000 7fe050c1c740 1 -- 172.21.15.33:0/1520216488 wait complete. 2024-01-25T16:12:02.569 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe050c1c740 20 client.15744 trim_cache size 0 max 16384 2024-01-25T16:12:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:12:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:12:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:12:02.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 msgr2=0x7fe0300a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 0x7fe0300a39b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009fd0 tx=0x7fe03c013820 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 msgr2=0x7fe04819dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe04000eb10 tx=0x7fe04000c2d0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 shutdown_connections 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe0300a14f0 0x7fe0300a39b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe048106dd0 0x7fe04819dbf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0480ff780 0x7fe048199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 --2- 172.21.15.33:0/465863337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe0480fedc0 0x7fe048199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 >> 172.21.15.33:0/465863337 conn(0x7fe048075d80 msgr2=0x7fe04810c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 shutdown_connections 2024-01-25T16:12:02.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:02.566+0000 7fe04e058640 1 -- 172.21.15.33:0/465863337 wait complete. 2024-01-25T16:12:02.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:02 smithi152 ceph-mon[36015]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-01-25T16:12:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:02 smithi076 ceph-mon[35642]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 1.1 MiB/s wr, 113 op/s 2024-01-25T16:12:02.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 83971 2.27 MB/sec execute 59 sec latency 662.346 ms 2024-01-25T16:12:03.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 83971 2.23 MB/sec execute 60 sec latency 1662.451 ms 2024-01-25T16:12:04.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 83971 2.20 MB/sec execute 61 sec latency 2662.507 ms 2024-01-25T16:12:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:05 smithi033 ceph-mon[31417]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 751 KiB/s wr, 73 op/s 2024-01-25T16:12:05.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:05 smithi152 ceph-mon[36015]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 751 KiB/s wr, 73 op/s 2024-01-25T16:12:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:05 smithi076 ceph-mon[35642]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 751 KiB/s wr, 73 op/s 2024-01-25T16:12:05.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84126 2.17 MB/sec execute 62 sec latency 3325.274 ms 2024-01-25T16:12:06.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84126 2.14 MB/sec execute 63 sec latency 1186.522 ms 2024-01-25T16:12:07.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:07 smithi033 ceph-mon[31417]: pgmap v131: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 999 KiB/s wr, 133 op/s 2024-01-25T16:12:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:07 smithi152 ceph-mon[36015]: pgmap v131: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 999 KiB/s wr, 133 op/s 2024-01-25T16:12:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:07 smithi076 ceph-mon[35642]: pgmap v131: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 598 KiB/s rd, 999 KiB/s wr, 133 op/s 2024-01-25T16:12:07.767 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84126 2.11 MB/sec execute 64 sec latency 2186.632 ms 2024-01-25T16:12:08.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84126 2.07 MB/sec execute 65 sec latency 3186.746 ms 2024-01-25T16:12:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:09 smithi033 ceph-mon[31417]: pgmap v132: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:09.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:09 smithi152 ceph-mon[36015]: pgmap v132: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:09 smithi076 ceph-mon[35642]: pgmap v132: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:09.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 84126 2.04 MB/sec execute 66 sec latency 4186.853 ms 2024-01-25T16:12:10.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 85161 2.16 MB/sec execute 67 sec latency 4860.892 ms 2024-01-25T16:12:11.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:11 smithi076 ceph-mon[35642]: pgmap v133: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:11 smithi033 ceph-mon[31417]: pgmap v133: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:11 smithi152 ceph-mon[36015]: pgmap v133: 97 pgs: 97 active+clean; 33 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 623 KiB/s wr, 107 op/s 2024-01-25T16:12:11.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87155 2.23 MB/sec execute 68 sec latency 19.542 ms 2024-01-25T16:12:12.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87512 2.22 MB/sec execute 69 sec latency 947.262 ms 2024-01-25T16:12:13.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:13 smithi076 ceph-mon[35642]: pgmap v134: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 974 KiB/s wr, 130 op/s 2024-01-25T16:12:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:13 smithi033 ceph-mon[31417]: pgmap v134: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 974 KiB/s wr, 130 op/s 2024-01-25T16:12:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:13 smithi152 ceph-mon[36015]: pgmap v134: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 974 KiB/s wr, 130 op/s 2024-01-25T16:12:13.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87512 2.19 MB/sec execute 70 sec latency 1947.351 ms 2024-01-25T16:12:14.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87512 2.16 MB/sec execute 71 sec latency 2947.437 ms 2024-01-25T16:12:15.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:15 smithi076 ceph-mon[35642]: pgmap v135: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 599 KiB/s wr, 83 op/s 2024-01-25T16:12:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:15 smithi033 ceph-mon[31417]: pgmap v135: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 599 KiB/s wr, 83 op/s 2024-01-25T16:12:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:15 smithi152 ceph-mon[36015]: pgmap v135: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 515 KiB/s rd, 599 KiB/s wr, 83 op/s 2024-01-25T16:12:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 -- 172.21.15.33:0/3727260675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe388104bb0 msgr2=0x7fe388104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/3727260675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe388104bb0 0x7fe388104f90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009900 tx=0x7fe37c02bc80 comp rx=0 tx=0).stop 2024-01-25T16:12:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 -- 172.21.15.33:0/3727260675 shutdown_connections 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/3727260675 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 0x7fe38810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/3727260675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3880ffe40 0x7fe3881002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/3727260675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe388104bb0 0x7fe388104f90 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 -- 172.21.15.33:0/3727260675 >> 172.21.15.33:0/3727260675 conn(0x7fe388076a50 msgr2=0x7fe388076e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 -- 172.21.15.33:0/3727260675 shutdown_connections 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.663+0000 7fe38ec9b640 1 -- 172.21.15.33:0/3727260675 wait complete. 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 Processor -- start 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 -- start start 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 0x7fe388196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe388104bb0 0x7fe38819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819afc0 con 0x7fe3880ffe40 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819b130 con 0x7fe388104bb0 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ec9b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819b2f0 con 0x7fe3881007e0 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52354/0 (socket says 172.21.15.33:52354) 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 -- 172.21.15.33:0/4233626707 learned_addr learned my addr 172.21.15.33:0/4233626707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38d238640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe388104bb0 0x7fe38819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe387fff640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 0x7fe388196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 -- 172.21.15.33:0/4233626707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 msgr2=0x7fe388196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 0x7fe388196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.664+0000 7fe38ca37640 1 -- 172.21.15.33:0/4233626707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe388104bb0 msgr2=0x7fe38819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe38ca37640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe388104bb0 0x7fe38819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe38ca37640 1 -- 172.21.15.33:0/4233626707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009560 con 0x7fe3880ffe40 2024-01-25T16:12:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe38ca37640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe37c005940 tx=0x7fe37c004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe385ffb640 1 -- 172.21.15.33:0/4233626707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c004350 con 0x7fe3880ffe40 2024-01-25T16:12:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe385ffb640 1 -- 172.21.15.33:0/4233626707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37c0044b0 con 0x7fe3880ffe40 2024-01-25T16:12:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe38819b570 con 0x7fe3880ffe40 2024-01-25T16:12:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe385ffb640 1 -- 172.21.15.33:0/4233626707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37c003e10 con 0x7fe3880ffe40 2024-01-25T16:12:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.665+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a2e50 con 0x7fe3880ffe40 2024-01-25T16:12:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe385ffb640 1 -- 172.21.15.33:0/4233626707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fe37c005000 con 0x7fe3880ffe40 2024-01-25T16:12:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 Processor -- start 2024-01-25T16:12:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 -- start start 2024-01-25T16:12:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 0x55cc23d73470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe385ffb640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 0x7fe3780a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d7c780 0x55cc23d7eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23ce2070 con 0x55cc23d739b0 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23cdf760 con 0x55cc23d73090 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe39185e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23cdb050 con 0x55cc23d7c780 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe38ca37640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 0x7fe3780a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52360/0 (socket says 172.21.15.33:52360) 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe387fff640 1 -- 172.21.15.33:0/1314649160 learned_addr learned my addr 172.21.15.33:0/1314649160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.668+0000 7fe385ffb640 1 -- 172.21.15.33:0/4233626707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe37c0ef6c0 con 0x7fe3880ffe40 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe38d238640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d7c780 0x55cc23d7eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe38d238640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 0x55cc23d73470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 -- 172.21.15.33:0/1314649160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d7c780 msgr2=0x55cc23d7eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d7c780 0x55cc23d7eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 -- 172.21.15.33:0/1314649160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 msgr2=0x55cc23d73470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 0x55cc23d73470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 -- 172.21.15.33:0/1314649160 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc23cd2e70 con 0x55cc23d739b0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe38d238640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 0x55cc23d73470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe38ca37640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 0x7fe3780a3a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0bc280 tx=0x7fe37c08bcf0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe387fff640 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe37400bc80 tx=0x7fe37400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f1e5c68a81aa20 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe372ffd640 1 -- 172.21.15.33:0/1314649160 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37401a460 con 0x55cc23d739b0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe372ffd640 1 -- 172.21.15.33:0/1314649160 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37401aa80 con 0x55cc23d739b0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.669+0000 7fe372ffd640 1 -- 172.21.15.33:0/1314649160 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3740119f0 con 0x55cc23d739b0 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- 172.21.15.33:0/1314649160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 msgr2=0x55cc23d7c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe37400bc80 tx=0x7fe37400f730 comp rx=0 tx=0).stop 2024-01-25T16:12:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- 172.21.15.33:0/1314649160 shutdown_connections 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d7c780 0x55cc23d7eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d7c240 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 --2- 172.21.15.33:0/1314649160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d73090 0x55cc23d73470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- 172.21.15.33:0/1314649160 >> 172.21.15.33:0/1314649160 conn(0x55cc23d6d9b0 msgr2=0x55cc23d6dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- 172.21.15.33:0/1314649160 shutdown_connections 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- 172.21.15.33:0/1314649160 wait complete. 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 -- start start 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.670+0000 7fe39185e740 1 Processor -- start 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d79730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23d7a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23c941e0 con 0x55cc23d739b0 2024-01-25T16:12:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23c93e80 con 0x55cc23d7c780 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23c590c0 con 0x55cc23d73090 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe39185e740 10 client.0 _ll_register_callbacks cb 0x55cc23c7eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52376/0 (socket says 172.21.15.33:52376) 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 -- 172.21.15.33:0/1773805471 learned_addr learned my addr 172.21.15.33:0/1773805471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe38d238640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d79730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe38d238640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23d7a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 -- 172.21.15.33:0/1773805471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 msgr2=0x55cc23d79730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d79730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 -- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 msgr2=0x55cc23d7a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23d7a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 -- 172.21.15.33:0/1773805471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37400b8e0 con 0x55cc23d739b0 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe38d238640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23d7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe387fff640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe374006010 tx=0x7fe37400bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27db67064b2485ea server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.671+0000 7fe367fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374020070 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe37401a670 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc23cd4040 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe37401a830 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 10 client.15756 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cc23cd96d0 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe374032020 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe3740046c0 con 0x55cc23d739b0 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 10 client.15756 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 20 client.15756 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23d8c530 0x55cc23d8e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 4 client.15756 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc23cd3d60 con 0x55cc23d8c530 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.672+0000 7fe38d238640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23d8c530 0x55cc23d8e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.673+0000 7fe367fff640 10 client.15756 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.673+0000 7fe38d238640 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23d8c530 0x55cc23d8e950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31e1325227d9382a server_cookie=9b002030de84cbb3 in_seq=0 out_seq=0 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.673+0000 7fe367fff640 1 -- 172.21.15.33:0/1773805471 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc23cd3d60 con 0x55cc23d8c530 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.673+0000 7fe367fff640 10 client.15756 handle_command_reply: tid=0 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.673+0000 7fe39185e740 1 client.15756 shutdown 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 msgr2=0x55cc23d79c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe374006010 tx=0x7fe37400bfa0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 shutdown_connections 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23d8c530 0x55cc23d8e950 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23d7a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe367fff640 0 client.15756 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d739b0 0x55cc23d79c70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 --2- 172.21.15.33:0/1773805471 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d79730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 >> 172.21.15.33:0/1773805471 conn(0x55cc23d6d9b0 msgr2=0x55cc23d6f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 shutdown_connections 2024-01-25T16:12:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 1 -- 172.21.15.33:0/1773805471 wait complete. 2024-01-25T16:12:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.674+0000 7fe39185e740 20 client.15756 trim_cache size 0 max 16384 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 Processor -- start 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 -- start start 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d7be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 0x55cc23d79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23d6e9e0 con 0x55cc23d7c780 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23d83da0 con 0x55cc23d739b0 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe39185e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23d6d860 con 0x55cc23d73090 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe38d238640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe38d238640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52386/0 (socket says 172.21.15.33:52386) 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.687+0000 7fe38d238640 1 -- 172.21.15.33:0/1820621943 learned_addr learned my addr 172.21.15.33:0/1820621943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d7be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe387fff640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 0x55cc23d79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 -- 172.21.15.33:0/1820621943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 msgr2=0x55cc23d7be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d7be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 -- 172.21.15.33:0/1820621943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 msgr2=0x55cc23d79fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 0x55cc23d79fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 -- 172.21.15.33:0/1820621943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc23c76060 con 0x55cc23d7c780 2024-01-25T16:12:15.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe387fff640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 0x55cc23d79fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe38d238640 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe380011700 tx=0x7fe380004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a7eda7b928bb11 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe3727fc640 1 -- 172.21.15.33:0/1820621943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380011d00 con 0x55cc23d7c780 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe3727fc640 1 -- 172.21.15.33:0/1820621943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe380011e60 con 0x55cc23d7c780 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.688+0000 7fe3727fc640 1 -- 172.21.15.33:0/1820621943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380013dc0 con 0x55cc23d7c780 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- 172.21.15.33:0/1820621943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 msgr2=0x55cc23d78000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe380011700 tx=0x7fe380004ab0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- 172.21.15.33:0/1820621943 shutdown_connections 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d7c780 0x55cc23d78000 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d739b0 0x55cc23d79fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- 172.21.15.33:0/1820621943 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d73090 0x55cc23d7be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- 172.21.15.33:0/1820621943 >> 172.21.15.33:0/1820621943 conn(0x55cc23d6d9b0 msgr2=0x55cc23d83660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- 172.21.15.33:0/1820621943 shutdown_connections 2024-01-25T16:12:15.696 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- 172.21.15.33:0/1820621943 wait complete. 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- start start 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 Processor -- start 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d739b0 0x55cc23ddddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23dd8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23dd94b0 con 0x55cc23d73090 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23dd9620 con 0x55cc23d7c780 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc23dd97e0 con 0x55cc23d739b0 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.689+0000 7fe39185e740 10 client.0 _ll_register_callbacks cb 0x7fe37400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52402/0 (socket says 172.21.15.33:52402) 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 -- 172.21.15.33:0/1222017965 learned_addr learned my addr 172.21.15.33:0/1222017965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe387fff640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d739b0 0x55cc23ddddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.697 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 -- 172.21.15.33:0/1222017965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d739b0 msgr2=0x55cc23ddddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d739b0 0x55cc23ddddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 -- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 msgr2=0x55cc23dd8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23dd8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 -- 172.21.15.33:0/1222017965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe38001a040 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe38d238640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe380005c10 tx=0x7fe380009ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7adb127f9df7729 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe367fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe380004b20 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7fe380004c80 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc23d726e0 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe380004e40 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe39185e740 10 client.15762 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cc23c9ba80 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.690+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc23d726e0 con 0x55cc23d73090 2024-01-25T16:12:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe380004200 con 0x55cc23d73090 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 10 client.15762 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 20 client.15762 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23de66e0 0x55cc23de8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 4 client.15762 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fe37401aa80 con 0x55cc23de66e0 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe387fff640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23de66e0 0x55cc23de8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe387fff640 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23de66e0 0x55cc23de8b00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5651c7b14cd4ecf3 server_cookie=bbb6a783ba87c5e5 in_seq=0 out_seq=0 2024-01-25T16:12:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.691+0000 7fe367fff640 10 client.15762 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe367fff640 1 -- 172.21.15.33:0/1222017965 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fe37401aa80 con 0x55cc23de66e0 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe367fff640 10 client.15762 handle_command_reply: tid=0 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 client.15762 shutdown 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 msgr2=0x55cc23ddd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe380005c10 tx=0x7fe380009ad0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.729 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 shutdown_connections 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc23de66e0 0x55cc23de8b00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc23d7c780 0x55cc23dd8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc23d739b0 0x55cc23ddddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 --2- 172.21.15.33:0/1222017965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc23d73090 0x55cc23ddd880 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe367fff640 0 client.15762 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 >> 172.21.15.33:0/1222017965 conn(0x55cc23d6d9b0 msgr2=0x55cc23d6c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 shutdown_connections 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 1 -- 172.21.15.33:0/1222017965 wait complete. 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.726+0000 7fe39185e740 20 client.15762 trim_cache size 0 max 16384 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:12:15.730 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:12:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:12:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:12:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 msgr2=0x7fe3780a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.731 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 0x7fe3780a3a30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0bc280 tx=0x7fe37c08bcf0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 msgr2=0x7fe388196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe37c005940 tx=0x7fe37c004e50 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 shutdown_connections 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fe3780a1570 0x7fe3780a3a30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe388104bb0 0x7fe38819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3881007e0 0x7fe388196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 --2- 172.21.15.33:0/4233626707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3880ffe40 0x7fe388196150 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 >> 172.21.15.33:0/4233626707 conn(0x7fe388076a50 msgr2=0x7fe3880fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 shutdown_connections 2024-01-25T16:12:15.732 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:15.727+0000 7fe38ec9b640 1 -- 172.21.15.33:0/4233626707 wait complete. 2024-01-25T16:12:15.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87566 2.13 MB/sec execute 72 sec latency 3615.408 ms 2024-01-25T16:12:16.768 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87566 2.10 MB/sec execute 73 sec latency 1295.186 ms 2024-01-25T16:12:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:17 smithi033 ceph-mon[31417]: pgmap v136: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 982 KiB/s wr, 124 op/s 2024-01-25T16:12:17.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:17 smithi152 ceph-mon[36015]: pgmap v136: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 982 KiB/s wr, 124 op/s 2024-01-25T16:12:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:17 smithi076 ceph-mon[35642]: pgmap v136: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 982 KiB/s wr, 124 op/s 2024-01-25T16:12:17.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87566 2.07 MB/sec execute 74 sec latency 2295.274 ms 2024-01-25T16:12:18.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87566 2.04 MB/sec execute 75 sec latency 3295.391 ms 2024-01-25T16:12:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:19 smithi033 ceph-mon[31417]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:19.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:19 smithi152 ceph-mon[36015]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:19 smithi076 ceph-mon[35642]: pgmap v137: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:19.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87566 2.02 MB/sec execute 76 sec latency 4295.477 ms 2024-01-25T16:12:20.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87606 1.99 MB/sec execute 77 sec latency 4959.529 ms 2024-01-25T16:12:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:21 smithi033 ceph-mon[31417]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:21 smithi152 ceph-mon[36015]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:21 smithi076 ceph-mon[35642]: pgmap v138: 97 pgs: 97 active+clean; 38 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-01-25T16:12:21.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87606 1.97 MB/sec execute 78 sec latency 1294.944 ms 2024-01-25T16:12:22.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87606 1.94 MB/sec execute 79 sec latency 2295.058 ms 2024-01-25T16:12:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:23 smithi033 ceph-mon[31417]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 794 KiB/s wr, 68 op/s 2024-01-25T16:12:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:23 smithi152 ceph-mon[36015]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 794 KiB/s wr, 68 op/s 2024-01-25T16:12:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:23 smithi076 ceph-mon[35642]: pgmap v139: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 794 KiB/s wr, 68 op/s 2024-01-25T16:12:23.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87606 1.92 MB/sec execute 80 sec latency 3295.162 ms 2024-01-25T16:12:24.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87606 1.89 MB/sec execute 81 sec latency 4295.271 ms 2024-01-25T16:12:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:25 smithi033 ceph-mon[31417]: pgmap v140: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 442 KiB/s wr, 45 op/s 2024-01-25T16:12:25.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:25 smithi152 ceph-mon[36015]: pgmap v140: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 442 KiB/s wr, 45 op/s 2024-01-25T16:12:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:25 smithi076 ceph-mon[35642]: pgmap v140: 97 pgs: 97 active+clean; 38 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 442 KiB/s wr, 45 op/s 2024-01-25T16:12:25.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87667 1.88 MB/sec execute 82 sec latency 4966.835 ms 2024-01-25T16:12:26.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87667 1.85 MB/sec execute 83 sec latency 1266.303 ms 2024-01-25T16:12:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:27 smithi033 ceph-mon[31417]: pgmap v141: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 732 KiB/s wr, 58 op/s 2024-01-25T16:12:27.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:27 smithi152 ceph-mon[36015]: pgmap v141: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 732 KiB/s wr, 58 op/s 2024-01-25T16:12:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:27 smithi076 ceph-mon[35642]: pgmap v141: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 732 KiB/s wr, 58 op/s 2024-01-25T16:12:27.769 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87667 1.83 MB/sec execute 84 sec latency 2266.379 ms 2024-01-25T16:12:28.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87667 1.81 MB/sec execute 85 sec latency 3266.435 ms 2024-01-25T16:12:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 -- 172.21.15.33:0/277771005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d410d550 msgr2=0x7f05d410f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 --2- 172.21.15.33:0/277771005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d410d550 0x7f05d410f940 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009ca0 tx=0x7f05c402c180 comp rx=0 tx=0).stop 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 -- 172.21.15.33:0/277771005 shutdown_connections 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 --2- 172.21.15.33:0/277771005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d410d550 0x7f05d410f940 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 --2- 172.21.15.33:0/277771005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d40739c0 0x7f05d410d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 --2- 172.21.15.33:0/277771005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 0x7f05d4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 -- 172.21.15.33:0/277771005 >> 172.21.15.33:0/277771005 conn(0x7f05d40fc780 msgr2=0x7f05d40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 -- 172.21.15.33:0/277771005 shutdown_connections 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.823+0000 7f05d913c640 1 -- 172.21.15.33:0/277771005 wait complete. 2024-01-25T16:12:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.824+0000 7f05d913c640 1 Processor -- start 2024-01-25T16:12:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.824+0000 7f05d913c640 1 -- start start 2024-01-25T16:12:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 0x7f05d4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d410d550 0x7f05d419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d419e200 con 0x7f05d40739c0 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d419e370 con 0x7f05d40730a0 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d913c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d419e530 con 0x7f05d410d550 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48268/0 (socket says 172.21.15.33:48268) 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 -- 172.21.15.33:0/2616177421 learned_addr learned my addr 172.21.15.33:0/2616177421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d359e640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d410d550 0x7f05d419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 0x7f05d4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 -- 172.21.15.33:0/2616177421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d410d550 msgr2=0x7f05d419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d410d550 0x7f05d419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 -- 172.21.15.33:0/2616177421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 msgr2=0x7f05d4199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 0x7f05d4199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 -- 172.21.15.33:0/2616177421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c4009900 con 0x7f05d40739c0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.825+0000 7f05d259c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00d770 tx=0x7f05bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.826+0000 7f05cbfff640 1 -- 172.21.15.33:0/2616177421 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc009c30 con 0x7f05d40739c0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.826+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d419e810 con 0x7f05d40739c0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.826+0000 7f05cbfff640 1 -- 172.21.15.33:0/2616177421 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05bc00de80 con 0x7f05d40739c0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.826+0000 7f05cbfff640 1 -- 172.21.15.33:0/2616177421 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc004de0 con 0x7f05d40739c0 2024-01-25T16:12:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.826+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d41a6090 con 0x7f05d40739c0 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.828+0000 7f05cbfff640 1 -- 172.21.15.33:0/2616177421 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f05bc004f40 con 0x7f05d40739c0 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.828+0000 7f05dbddc740 1 Processor -- start 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.828+0000 7f05dbddc740 1 -- start start 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05cbfff640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 0x7f05c00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d359e640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 0x7f05c00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05cbfff640 1 -- 172.21.15.33:0/2616177421 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05bc0cc8a0 con 0x7f05d40739c0 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696f2780 0x55dc696f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc69658070 con 0x55dc696e9090 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc69655760 con 0x55dc696f2780 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05dbddc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc69651050 con 0x55dc696e99b0 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48284/0 (socket says 172.21.15.33:48284) 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1956210177 learned_addr learned my addr 172.21.15.33:0/1956210177 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d259c640 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d359e640 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696f2780 0x55dc696f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d359e640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 0x7f05c00a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05c4004af0 tx=0x7f05c4005c50 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1956210177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 msgr2=0x55dc696f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1956210177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696f2780 msgr2=0x55dc696f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696f2780 0x55dc696f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.829+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1956210177 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc69648e70 con 0x55dc696e9090 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00bd10 tx=0x7f05cc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2909a8f34e3fdca2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05c97fa640 1 -- 172.21.15.33:0/1956210177 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc004030 con 0x55dc696e9090 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05c97fa640 1 -- 172.21.15.33:0/1956210177 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05cc015070 con 0x55dc696e9090 2024-01-25T16:12:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05c97fa640 1 -- 172.21.15.33:0/1956210177 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc0116a0 con 0x55dc696e9090 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05dbddc740 1 -- 172.21.15.33:0/1956210177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 msgr2=0x55dc696e9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.830+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f05cc00bd10 tx=0x7f05cc0107a0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- 172.21.15.33:0/1956210177 shutdown_connections 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696f2780 0x55dc696f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1956210177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696e9470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- 172.21.15.33:0/1956210177 >> 172.21.15.33:0/1956210177 conn(0x55dc696e39b0 msgr2=0x55dc696e3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- 172.21.15.33:0/1956210177 shutdown_connections 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- 172.21.15.33:0/1956210177 wait complete. 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- start start 2024-01-25T16:12:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 Processor -- start 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e99b0 0x55dc696efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc696f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6960a1e0 con 0x55dc696e9090 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc69609e80 con 0x55dc696e99b0 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc695cf0c0 con 0x55dc696f2780 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05dbddc740 10 client.0 _ll_register_callbacks cb 0x55dc695f4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.831+0000 7f05d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48300/0 (socket says 172.21.15.33:48300) 2024-01-25T16:12:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1572658331 learned_addr learned my addr 172.21.15.33:0/1572658331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d359e640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc696f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d259c640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e99b0 0x55dc696efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1572658331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 msgr2=0x55dc696f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc696f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e99b0 msgr2=0x55dc696efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e99b0 0x55dc696efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 -- 172.21.15.33:0/1572658331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc00b940 con 0x55dc696e9090 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0058e0 tx=0x7f05cc0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a622185b78695b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc020070 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05cc017070 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc6964a040 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05cc005040 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05dbddc740 10 client.15774 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.832+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc6964f6d0 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05cc031020 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f05cc0046c0 con 0x55dc696e9090 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 10 client.15774 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 20 client.15774 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc69702470 0x55dc69704880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 4 client.15774 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc69649d60 con 0x55dc69702470 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.833+0000 7f05d259c640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc69702470 0x55dc69704880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05ba7fc640 10 client.15774 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05d259c640 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc69702470 0x55dc69704880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70da0d7ab3ef50e9 server_cookie=f1878131284b350c in_seq=0 out_seq=0 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05ba7fc640 1 -- 172.21.15.33:0/1572658331 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc69649d60 con 0x55dc69702470 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05ba7fc640 10 client.15774 handle_command_reply: tid=0 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 client.15774 shutdown 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 msgr2=0x55dc696ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0058e0 tx=0x7f05cc0035f0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 shutdown_connections 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc69702470 0x55dc69704880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc696f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e99b0 0x55dc696efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05ba7fc640 0 client.15774 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1572658331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e9090 0x55dc696ef6d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 >> 172.21.15.33:0/1572658331 conn(0x55dc696e39b0 msgr2=0x55dc696e5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.834+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 shutdown_connections 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.835+0000 7f05dbddc740 1 -- 172.21.15.33:0/1572658331 wait complete. 2024-01-25T16:12:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.835+0000 7f05dbddc740 20 client.15774 trim_cache size 0 max 16384 2024-01-25T16:12:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 Processor -- start 2024-01-25T16:12:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 -- start start 2024-01-25T16:12:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc696e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc69649d60 con 0x55dc696f2780 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc696ddf50 con 0x55dc696e9090 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05dbddc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc696ea5a0 con 0x55dc696e99b0 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05d359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05d359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48302/0 (socket says 172.21.15.33:48302) 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05d359e640 1 -- 172.21.15.33:0/1259068959 learned_addr learned my addr 172.21.15.33:0/1259068959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.848+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc696e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d259c640 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 -- 172.21.15.33:0/1259068959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 msgr2=0x55dc696f9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 -- 172.21.15.33:0/1259068959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 msgr2=0x55dc696e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc696e54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 -- 172.21.15.33:0/1259068959 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc695ec060 con 0x55dc696f2780 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05d359e640 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009ca0 tx=0x7f05c4044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bd1bbb59dd07ba server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05c8ff9640 1 -- 172.21.15.33:0/1259068959 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c40328b0 con 0x55dc696f2780 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05c8ff9640 1 -- 172.21.15.33:0/1259068959 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05c4056460 con 0x55dc696f2780 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.849+0000 7f05c8ff9640 1 -- 172.21.15.33:0/1259068959 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4032ed0 con 0x55dc696f2780 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- 172.21.15.33:0/1259068959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 msgr2=0x55dc696f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009ca0 tx=0x7f05c4044000 comp rx=0 tx=0).stop 2024-01-25T16:12:28.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- 172.21.15.33:0/1259068959 shutdown_connections 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696f2780 0x55dc696f99b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696e99b0 0x55dc696f9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- 172.21.15.33:0/1259068959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc696e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- 172.21.15.33:0/1259068959 >> 172.21.15.33:0/1259068959 conn(0x55dc696e39b0 msgr2=0x55dc696e4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- 172.21.15.33:0/1259068959 shutdown_connections 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- 172.21.15.33:0/1259068959 wait complete. 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- start start 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 Processor -- start 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc697527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.859 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc6974cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6974d1b0 con 0x55dc696e99b0 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6974d320 con 0x55dc696e9090 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc6974d4e0 con 0x55dc696f2780 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.850+0000 7f05dbddc740 10 client.0 _ll_register_callbacks cb 0x7f05cc00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48308/0 (socket says 172.21.15.33:48308) 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 -- 172.21.15.33:0/2403681756 learned_addr learned my addr 172.21.15.33:0/2403681756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc697527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d359e640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc6974cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 -- 172.21.15.33:0/2403681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 msgr2=0x55dc6974cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc6974cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 -- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 msgr2=0x55dc697527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc697527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 -- 172.21.15.33:0/2403681756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d41105f0 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05d259c640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05bc099460 tx=0x7f05bc0a1b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4036fc5ab5ead4 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc009e50 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7f05bc0ca280 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f05cc015070 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05bc0ca440 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05dbddc740 10 client.15780 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55dc69611a80 con 0x55dc696e99b0 2024-01-25T16:12:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.851+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05cc015070 con 0x55dc696e99b0 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f05bc07f070 con 0x55dc696e99b0 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 10 client.15780 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 20 client.15780 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc6975a710 0x55dc6975cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 4 client.15780 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dc69350600 con 0x55dc6975a710 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc6975a710 0x55dc6975cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05ba7fc640 10 client.15780 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.852+0000 7f05d2d9d640 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc6975a710 0x55dc6975cb30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d66b174e47e5549f server_cookie=8b22dbfa8ea7202c in_seq=0 out_seq=0 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.873+0000 7f05ba7fc640 1 -- 172.21.15.33:0/2403681756 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dc69350600 con 0x55dc6975a710 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.873+0000 7f05ba7fc640 10 client.15780 handle_command_reply: tid=0 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.873+0000 7f05dbddc740 1 client.15780 shutdown 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.873+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 msgr2=0x55dc69752d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.873+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05bc099460 tx=0x7f05bc0a1b00 comp rx=0 tx=0).stop 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 shutdown_connections 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55dc6975a710 0x55dc6975cb30 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55dc696f2780 0x55dc6974cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc696e99b0 0x55dc69752d20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05ba7fc640 0 client.15780 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 --2- 172.21.15.33:0/2403681756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dc696e9090 0x55dc697527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 >> 172.21.15.33:0/2403681756 conn(0x55dc696e39b0 msgr2=0x55dc696e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 shutdown_connections 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 1 -- 172.21.15.33:0/2403681756 wait complete. 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05dbddc740 20 client.15780 trim_cache size 0 max 16384 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:12:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:12:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:12:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 msgr2=0x7f05c00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 0x7f05c00a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f05c4004af0 tx=0x7f05c4005c50 comp rx=0 tx=0).stop 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 msgr2=0x7f05d41998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00d770 tx=0x7f05bc00dc30 comp rx=0 tx=0).stop 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 shutdown_connections 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f05c00a14f0 0x7f05c00a39b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f05d410d550 0x7f05d419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05d40739c0 0x7f05d41998d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 --2- 172.21.15.33:0/2616177421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05d40730a0 0x7f05d4199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 >> 172.21.15.33:0/2616177421 conn(0x7f05d40fc780 msgr2=0x7f05d40fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.874+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 shutdown_connections 2024-01-25T16:12:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:28.875+0000 7f05d913c640 1 -- 172.21.15.33:0/2616177421 wait complete. 2024-01-25T16:12:29.471 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:29 smithi076 ceph-mon[35642]: pgmap v142: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:29 smithi033 ceph-mon[31417]: pgmap v142: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:29.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:29 smithi152 ceph-mon[36015]: pgmap v142: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:29.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 87667 1.79 MB/sec execute 86 sec latency 4266.528 ms 2024-01-25T16:12:30.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88723 1.88 MB/sec execute 87 sec latency 4928.996 ms 2024-01-25T16:12:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:31 smithi033 ceph-mon[31417]: pgmap v143: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:31.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:31 smithi152 ceph-mon[36015]: pgmap v143: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:31 smithi076 ceph-mon[35642]: pgmap v143: 97 pgs: 97 active+clean; 39 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 348 KiB/s wr, 17 op/s 2024-01-25T16:12:31.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91061 1.96 MB/sec execute 88 sec latency 51.615 ms 2024-01-25T16:12:32.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91061 1.93 MB/sec execute 89 sec latency 1051.706 ms 2024-01-25T16:12:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:33 smithi033 ceph-mon[31417]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 446 KiB/s wr, 39 op/s 2024-01-25T16:12:33.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:33 smithi152 ceph-mon[36015]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 446 KiB/s wr, 39 op/s 2024-01-25T16:12:33.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:33 smithi076 ceph-mon[35642]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 446 KiB/s wr, 39 op/s 2024-01-25T16:12:33.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91061 1.91 MB/sec execute 90 sec latency 2051.778 ms 2024-01-25T16:12:34.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91061 1.89 MB/sec execute 91 sec latency 3051.868 ms 2024-01-25T16:12:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:35 smithi033 ceph-mon[31417]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 387 KiB/s wr, 34 op/s 2024-01-25T16:12:35.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:35 smithi152 ceph-mon[36015]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 387 KiB/s wr, 34 op/s 2024-01-25T16:12:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:35 smithi076 ceph-mon[35642]: pgmap v145: 97 pgs: 97 active+clean; 36 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 387 KiB/s wr, 34 op/s 2024-01-25T16:12:35.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 91707 1.93 MB/sec execute 92 sec latency 3730.540 ms 2024-01-25T16:12:36.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 93799 2.04 MB/sec execute 93 sec latency 12.496 ms 2024-01-25T16:12:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:37 smithi033 ceph-mon[31417]: pgmap v146: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 671 KiB/s wr, 68 op/s 2024-01-25T16:12:37.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:37 smithi152 ceph-mon[36015]: pgmap v146: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 671 KiB/s wr, 68 op/s 2024-01-25T16:12:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:37 smithi076 ceph-mon[35642]: pgmap v146: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 671 KiB/s wr, 68 op/s 2024-01-25T16:12:37.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 94660 2.04 MB/sec execute 94 sec latency 801.596 ms 2024-01-25T16:12:38.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 94660 2.01 MB/sec execute 95 sec latency 1801.664 ms 2024-01-25T16:12:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:39 smithi033 ceph-mon[31417]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:39 smithi152 ceph-mon[36015]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:39 smithi076 ceph-mon[35642]: pgmap v147: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:39.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 94700 2.00 MB/sec execute 96 sec latency 2751.124 ms 2024-01-25T16:12:40.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 95646 2.05 MB/sec execute 97 sec latency 670.976 ms 2024-01-25T16:12:41.418 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:41 smithi076 ceph-mon[35642]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:41 smithi033 ceph-mon[31417]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:41.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:41 smithi152 ceph-mon[36015]: pgmap v148: 97 pgs: 97 active+clean; 38 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 382 KiB/s wr, 55 op/s 2024-01-25T16:12:41.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 98185 2.15 MB/sec execute 98 sec latency 10.083 ms 2024-01-25T16:12:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 -- 172.21.15.33:0/2196787384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff34101ed0 msgr2=0x7eff341022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 --2- 172.21.15.33:0/2196787384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff34101ed0 0x7eff341022b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7eff24009900 tx=0x7eff2402bc30 comp rx=0 tx=0).stop 2024-01-25T16:12:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 -- 172.21.15.33:0/2196787384 shutdown_connections 2024-01-25T16:12:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 --2- 172.21.15.33:0/2196787384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff3410b340 0x7eff3410d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 --2- 172.21.15.33:0/2196787384 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff3410acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 --2- 172.21.15.33:0/2196787384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff34101ed0 0x7eff341022b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 -- 172.21.15.33:0/2196787384 >> 172.21.15.33:0/2196787384 conn(0x7eff34076d20 msgr2=0x7eff34077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 -- 172.21.15.33:0/2196787384 shutdown_connections 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.960+0000 7eff39f7d640 1 -- 172.21.15.33:0/2196787384 wait complete. 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 Processor -- start 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 -- start start 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff34101ed0 0x7eff341970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff34197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3419bfb0 con 0x7eff3410b340 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3419c120 con 0x7eff34101ed0 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff39f7d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3419c2e0 con 0x7eff341027f0 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44862/0 (socket says 172.21.15.33:44862) 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff33fff640 1 -- 172.21.15.33:0/3323600850 learned_addr learned my addr 172.21.15.33:0/3323600850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff337fe640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff34101ed0 0x7eff341970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.961+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff34197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 -- 172.21.15.33:0/3323600850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 msgr2=0x7eff34197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff34197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 -- 172.21.15.33:0/3323600850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff34101ed0 msgr2=0x7eff341970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff34101ed0 0x7eff341970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 -- 172.21.15.33:0/3323600850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff24009560 con 0x7eff3410b340 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff34197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff33fff640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00eb10 tx=0x7eff2c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff30ff9640 1 -- 172.21.15.33:0/3323600850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c004b20 con 0x7eff3410b340 2024-01-25T16:12:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff30ff9640 1 -- 172.21.15.33:0/3323600850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2c004c80 con 0x7eff3410b340 2024-01-25T16:12:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3419c5c0 con 0x7eff3410b340 2024-01-25T16:12:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff30ff9640 1 -- 172.21.15.33:0/3323600850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c010690 con 0x7eff3410b340 2024-01-25T16:12:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.962+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341a3e40 con 0x7eff3410b340 2024-01-25T16:12:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.964+0000 7eff30ff9640 1 -- 172.21.15.33:0/3323600850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7eff2c005020 con 0x7eff3410b340 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.964+0000 7eff3cc14740 1 Processor -- start 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 -- start start 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d41398470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff30ff9640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 0x7eff1c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413a1780 0x563d413a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41307070 con 0x563d413989b0 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41304760 con 0x563d41398090 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff3cc14740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41300050 con 0x563d413a1780 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44872/0 (socket says 172.21.15.33:44872) 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 -- 172.21.15.33:0/316265331 learned_addr learned my addr 172.21.15.33:0/316265331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff33fff640 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413a1780 0x563d413a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 0x7eff1c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff30ff9640 1 -- 172.21.15.33:0/3323600850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff2c014070 con 0x7eff3410b340 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff32ffd640 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d41398470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 -- 172.21.15.33:0/316265331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413a1780 msgr2=0x563d413a3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413a1780 0x563d413a3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 -- 172.21.15.33:0/316265331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 msgr2=0x563d41398470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d41398470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 -- 172.21.15.33:0/316265331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d412f7e70 con 0x563d413989b0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 0x7eff1c0a39b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff340ff420 tx=0x7eff2800b7e0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.965+0000 7eff337fe640 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7eff24005ab0 tx=0x7eff24005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bfb194ef48b18f7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff19ffb640 1 -- 172.21.15.33:0/316265331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240048c0 con 0x563d413989b0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff19ffb640 1 -- 172.21.15.33:0/316265331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff24004a20 con 0x563d413989b0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff19ffb640 1 -- 172.21.15.33:0/316265331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24005290 con 0x563d413989b0 2024-01-25T16:12:41.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 -- 172.21.15.33:0/316265331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 msgr2=0x563d413a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7eff24005ab0 tx=0x7eff24005e80 comp rx=0 tx=0).stop 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 -- 172.21.15.33:0/316265331 shutdown_connections 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413a1780 0x563d413a3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a1240 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 --2- 172.21.15.33:0/316265331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d41398470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 -- 172.21.15.33:0/316265331 >> 172.21.15.33:0/316265331 conn(0x563d413929b0 msgr2=0x563d41392d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 -- 172.21.15.33:0/316265331 shutdown_connections 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.966+0000 7eff3cc14740 1 -- 172.21.15.33:0/316265331 wait complete. 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 -- start start 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 Processor -- start 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d4139e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d4139f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d412b91e0 con 0x563d413989b0 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d412b8e80 con 0x563d413a1780 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4127e0c0 con 0x563d41398090 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff3cc14740 10 client.0 _ll_register_callbacks cb 0x563d412a3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44888/0 (socket says 172.21.15.33:44888) 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 -- 172.21.15.33:0/3760027659 learned_addr learned my addr 172.21.15.33:0/3760027659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff337fe640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d4139e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff33fff640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d4139f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 -- 172.21.15.33:0/3760027659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 msgr2=0x563d4139e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d4139e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 -- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 msgr2=0x563d4139f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d4139f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff32ffd640 1 -- 172.21.15.33:0/3760027659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3410e430 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.967+0000 7eff337fe640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d4139e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff32ffd640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7eff280038f0 tx=0x7eff28005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0e9b07b391c013 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28003e70 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff28010ad0 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d412f9040 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff28010c90 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 10 client.15792 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563d412fe6d0 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563d412f9040 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff28004a70 con 0x563d413989b0 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 10 client.15792 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 20 client.15792 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d413b3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 4 client.15792 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.968+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d412f8d60 con 0x563d413b1520 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.969+0000 7eff337fe640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d413b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.969+0000 7eff337fe640 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d413b3960 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509a95ef676a3f02 server_cookie=30b4d8ab9f271955 in_seq=0 out_seq=0 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.969+0000 7eff12ffd640 10 client.15792 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff12ffd640 1 -- 172.21.15.33:0/3760027659 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d412f8d60 con 0x563d413b1520 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff12ffd640 10 client.15792 handle_command_reply: tid=0 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 client.15792 shutdown 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 msgr2=0x563d4139ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7eff280038f0 tx=0x7eff28005f90 comp rx=0 tx=0).stop 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 shutdown_connections 2024-01-25T16:12:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d413b3960 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d4139f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d4139ec00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff12ffd640 0 client.15792 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3760027659 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d4139e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 >> 172.21.15.33:0/3760027659 conn(0x563d413929b0 msgr2=0x563d41394700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 shutdown_connections 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 1 -- 172.21.15.33:0/3760027659 wait complete. 2024-01-25T16:12:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.970+0000 7eff3cc14740 20 client.15792 trim_cache size 0 max 16384 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 Processor -- start 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 -- start start 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d413a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413989b0 0x563d4139ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d4138e0e0 con 0x563d413a1780 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d412f8d90 con 0x563d41398090 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff3cc14740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41392840 con 0x563d413989b0 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44898/0 (socket says 172.21.15.33:44898) 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 -- 172.21.15.33:0/3165639062 learned_addr learned my addr 172.21.15.33:0/3165639062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff337fe640 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d413a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 -- 172.21.15.33:0/3165639062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413989b0 msgr2=0x563d4139ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413989b0 0x563d4139ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 -- 172.21.15.33:0/3165639062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 msgr2=0x563d413a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d413a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.983+0000 7eff33fff640 1 -- 172.21.15.33:0/3165639062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d4129b060 con 0x563d413a1780 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff33fff640 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7eff2c09a3b0 tx=0x7eff2c009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c449a280eb741479 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff197fa640 1 -- 172.21.15.33:0/3165639062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c09ab20 con 0x563d413a1780 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff197fa640 1 -- 172.21.15.33:0/3165639062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2c004df0 con 0x563d413a1780 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff197fa640 1 -- 172.21.15.33:0/3165639062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0a27b0 con 0x563d413a1780 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 -- 172.21.15.33:0/3165639062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 msgr2=0x563d4139b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7eff2c09a3b0 tx=0x7eff2c009a70 comp rx=0 tx=0).stop 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 -- 172.21.15.33:0/3165639062 shutdown_connections 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413a1780 0x563d4139b940 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d413989b0 0x563d4139ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 --2- 172.21.15.33:0/3165639062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d41398090 0x563d413a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 -- 172.21.15.33:0/3165639062 >> 172.21.15.33:0/3165639062 conn(0x563d413929b0 msgr2=0x563d41398f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 -- 172.21.15.33:0/3165639062 shutdown_connections 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.984+0000 7eff3cc14740 1 -- 172.21.15.33:0/3165639062 wait complete. 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 -- start start 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 Processor -- start 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d413a6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d41406a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41406fc0 con 0x563d413989b0 2024-01-25T16:12:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41407100 con 0x563d413a1780 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d41407270 con 0x563d41398090 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff3cc14740 10 client.0 _ll_register_callbacks cb 0x7eff2c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44902/0 (socket says 172.21.15.33:44902) 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 -- 172.21.15.33:0/2424244481 learned_addr learned my addr 172.21.15.33:0/2424244481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff33fff640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d41406a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff337fe640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d413a6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 -- 172.21.15.33:0/2424244481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 msgr2=0x563d413a6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d413a6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 -- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 msgr2=0x563d41406a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d41406a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.985+0000 7eff32ffd640 1 -- 172.21.15.33:0/2424244481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341a3e40 con 0x563d413989b0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff32ffd640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff28004dd0 tx=0x7eff2801e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c7c19c6bd3554 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff28003e70 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8498+0+0 (secure 0 0 0) 0x7eff280047a0 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff24004a20 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff28004960 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 10 client.15798 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563d412c0a80 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff24004a20 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7eff2801ea20 con 0x563d413989b0 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 10 client.15798 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 20 client.15798 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d4140aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 4 client.15798 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.986+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d40fff600 con 0x563d413b1520 2024-01-25T16:12:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.987+0000 7eff337fe640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d4140aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.987+0000 7eff337fe640 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d4140aa50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a165dab1c65cbe8d server_cookie=2fc970d1e4f9579 in_seq=0 out_seq=0 2024-01-25T16:12:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:41.987+0000 7eff12ffd640 10 client.15798 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:42.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.040+0000 7eff12ffd640 1 -- 172.21.15.33:0/2424244481 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d40fff600 con 0x563d413b1520 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.040+0000 7eff12ffd640 10 client.15798 handle_command_reply: tid=0 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.040+0000 7eff3cc14740 1 client.15798 shutdown 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.040+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 msgr2=0x563d413a6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff28004dd0 tx=0x7eff2801e670 comp rx=0 tx=0).stop 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 shutdown_connections 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563d413b1520 0x563d4140aa50 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563d413a1780 0x563d41406a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d413989b0 0x563d413a6c80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff12ffd640 0 client.15798 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 --2- 172.21.15.33:0/2424244481 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563d41398090 0x563d413a6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 >> 172.21.15.33:0/2424244481 conn(0x563d413929b0 msgr2=0x563d4139cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 shutdown_connections 2024-01-25T16:12:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 1 -- 172.21.15.33:0/2424244481 wait complete. 2024-01-25T16:12:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff3cc14740 20 client.15798 trim_cache size 0 max 16384 2024-01-25T16:12:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:12:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:12:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:12:42.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 msgr2=0x7eff1c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 0x7eff1c0a39b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff340ff420 tx=0x7eff2800b7e0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 msgr2=0x7eff3419b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00eb10 tx=0x7eff2c00c2d0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 shutdown_connections 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7eff1c0a14f0 0x7eff1c0a39b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff3410b340 0x7eff3419b9b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff341027f0 0x7eff34197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 --2- 172.21.15.33:0/3323600850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff34101ed0 0x7eff341970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.041+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 >> 172.21.15.33:0/3323600850 conn(0x7eff34076d20 msgr2=0x7eff340fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.042+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 shutdown_connections 2024-01-25T16:12:42.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:42.042+0000 7eff39f7d640 1 -- 172.21.15.33:0/3323600850 wait complete. 2024-01-25T16:12:42.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 98246 2.13 MB/sec execute 99 sec latency 950.768 ms 2024-01-25T16:12:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:43 smithi033 ceph-mon[31417]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 689 KiB/s wr, 91 op/s 2024-01-25T16:12:43.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:43 smithi152 ceph-mon[36015]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 689 KiB/s wr, 91 op/s 2024-01-25T16:12:43.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:43 smithi076 ceph-mon[35642]: pgmap v149: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 689 KiB/s wr, 91 op/s 2024-01-25T16:12:43.770 INFO:tasks.workunit.client.0.smithi033.stdout: 1 98246 2.11 MB/sec execute 100 sec latency 1950.869 ms 2024-01-25T16:12:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:44 smithi033 ceph-mon[31417]: evicting unresponsive client smithi033:x (15294), after 300.924 seconds 2024-01-25T16:12:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:44 smithi033 ceph-mon[31417]: Evicting (and blocklisting) client session 15294 (172.21.15.33:0/1587480808) 2024-01-25T16:12:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:44 smithi033 ceph-mon[31417]: from='mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777]' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:44 smithi033 ceph-mon[31417]: from='mds.? ' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:44 smithi152 ceph-mon[36015]: evicting unresponsive client smithi033:x (15294), after 300.924 seconds 2024-01-25T16:12:44.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:44 smithi152 ceph-mon[36015]: Evicting (and blocklisting) client session 15294 (172.21.15.33:0/1587480808) 2024-01-25T16:12:44.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:44 smithi152 ceph-mon[36015]: from='mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777]' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:44 smithi152 ceph-mon[36015]: from='mds.? ' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:44 smithi076 ceph-mon[35642]: evicting unresponsive client smithi033:x (15294), after 300.924 seconds 2024-01-25T16:12:44.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:44 smithi076 ceph-mon[35642]: Evicting (and blocklisting) client session 15294 (172.21.15.33:0/1587480808) 2024-01-25T16:12:44.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:44 smithi076 ceph-mon[35642]: from='mds.? [v2:172.21.15.152:6838/2298219777,v1:172.21.15.152:6839/2298219777]' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:44 smithi076 ceph-mon[35642]: from='mds.? ' entity='mds.i' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]: dispatch 2024-01-25T16:12:44.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 98307 2.09 MB/sec execute 101 sec latency 2393.715 ms 2024-01-25T16:12:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:45 smithi033 ceph-mon[31417]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 69 op/s 2024-01-25T16:12:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:45 smithi033 ceph-mon[31417]: from='mds.? ' entity='mds.i' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]': finished 2024-01-25T16:12:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:45 smithi033 ceph-mon[31417]: osdmap e90: 12 total, 12 up, 12 in 2024-01-25T16:12:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:45 smithi152 ceph-mon[36015]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 69 op/s 2024-01-25T16:12:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:45 smithi152 ceph-mon[36015]: from='mds.? ' entity='mds.i' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]': finished 2024-01-25T16:12:45.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:45 smithi152 ceph-mon[36015]: osdmap e90: 12 total, 12 up, 12 in 2024-01-25T16:12:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:45 smithi076 ceph-mon[35642]: pgmap v150: 97 pgs: 97 active+clean; 40 MiB data, 608 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 592 KiB/s wr, 69 op/s 2024-01-25T16:12:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:45 smithi076 ceph-mon[35642]: from='mds.? ' entity='mds.i' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"172.21.15.33:0/1587480808"}]': finished 2024-01-25T16:12:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:45 smithi076 ceph-mon[35642]: osdmap e90: 12 total, 12 up, 12 in 2024-01-25T16:12:45.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 99373 2.17 MB/sec execute 102 sec latency 1169.016 ms 2024-01-25T16:12:46.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101750 2.23 MB/sec execute 103 sec latency 24.226 ms 2024-01-25T16:12:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:47 smithi033 ceph-mon[31417]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:47 smithi152 ceph-mon[36015]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:47 smithi076 ceph-mon[35642]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:47.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101750 2.21 MB/sec execute 104 sec latency 1024.294 ms 2024-01-25T16:12:48.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101750 2.19 MB/sec execute 105 sec latency 2024.410 ms 2024-01-25T16:12:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:49 smithi033 ceph-mon[31417]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:49.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:49 smithi152 ceph-mon[36015]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:49 smithi076 ceph-mon[35642]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:49.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101750 2.17 MB/sec execute 106 sec latency 3024.467 ms 2024-01-25T16:12:50.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101851 2.15 MB/sec execute 107 sec latency 3695.954 ms 2024-01-25T16:12:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:51 smithi033 ceph-mon[31417]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:51.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:51 smithi152 ceph-mon[36015]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:51 smithi076 ceph-mon[35642]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 934 KiB/s wr, 144 op/s 2024-01-25T16:12:51.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101851 2.13 MB/sec execute 108 sec latency 1213.075 ms 2024-01-25T16:12:52.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101851 2.11 MB/sec execute 109 sec latency 2213.212 ms 2024-01-25T16:12:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:53 smithi033 ceph-mon[31417]: pgmap v155: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:53.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:53 smithi152 ceph-mon[36015]: pgmap v155: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:53 smithi076 ceph-mon[35642]: pgmap v155: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:53.771 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101851 2.09 MB/sec execute 110 sec latency 3213.318 ms 2024-01-25T16:12:54.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101851 2.08 MB/sec execute 111 sec latency 4213.441 ms 2024-01-25T16:12:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 -- 172.21.15.33:0/183533764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 msgr2=0x7f26e01002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 --2- 172.21.15.33:0/183533764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e01002a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009900 tx=0x7f26c802bc30 comp rx=0 tx=0).stop 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 -- 172.21.15.33:0/183533764 shutdown_connections 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 --2- 172.21.15.33:0/183533764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 0x7f26e010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 --2- 172.21.15.33:0/183533764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e01002a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 --2- 172.21.15.33:0/183533764 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 0x7f26e0104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 -- 172.21.15.33:0/183533764 >> 172.21.15.33:0/183533764 conn(0x7f26e00769c0 msgr2=0x7f26e0076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 -- 172.21.15.33:0/183533764 shutdown_connections 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.139+0000 7f26e597d640 1 -- 172.21.15.33:0/183533764 wait complete. 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.140+0000 7f26e597d640 1 Processor -- start 2024-01-25T16:12:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.140+0000 7f26e597d640 1 -- start start 2024-01-25T16:12:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.140+0000 7f26e597d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 0x7f26e0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 0x7f26e019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e019e310 con 0x7f26e00ffe40 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e019e480 con 0x7f26e01007e0 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e019e640 con 0x7f26e0104b70 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54046/0 (socket says 172.21.15.33:54046) 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 -- 172.21.15.33:0/2368375062 learned_addr learned my addr 172.21.15.33:0/2368375062 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26df7fe640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 0x7f26e019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26de7fc640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 0x7f26e0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 -- 172.21.15.33:0/2368375062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 msgr2=0x7f26e019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 0x7f26e019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 -- 172.21.15.33:0/2368375062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 msgr2=0x7f26e0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 0x7f26e0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 -- 172.21.15.33:0/2368375062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8009560 con 0x7f26e00ffe40 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26deffd640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26d000ea80 tx=0x7f26d000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e497b640 1 -- 172.21.15.33:0/2368375062 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d000ce40 con 0x7f26e00ffe40 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e497b640 1 -- 172.21.15.33:0/2368375062 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d0005020 con 0x7f26e00ffe40 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e019e920 con 0x7f26e00ffe40 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e497b640 1 -- 172.21.15.33:0/2368375062 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d00116d0 con 0x7f26e00ffe40 2024-01-25T16:12:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.141+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e01a61a0 con 0x7f26e00ffe40 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.144+0000 7f26e497b640 1 -- 172.21.15.33:0/2368375062 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f26d0005190 con 0x7f26e00ffe40 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.144+0000 7f26e8616740 1 Processor -- start 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.144+0000 7f26e8616740 1 -- start start 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.144+0000 7f26e497b640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 0x7f26cc0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26df7fe640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 0x7f26cc0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3eeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e497b640 1 -- 172.21.15.33:0/2368375062 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f26d00cd120 con 0x7f26e00ffe40 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b352070 con 0x55cc1b3e39b0 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b34f760 con 0x55cc1b3e3090 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26e8616740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b34b050 con 0x55cc1b3ec780 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54048/0 (socket says 172.21.15.33:54048) 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 -- 172.21.15.33:0/1585795727 learned_addr learned my addr 172.21.15.33:0/1585795727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26df7fe640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3eeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26deffd640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 -- 172.21.15.33:0/1585795727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 msgr2=0x55cc1b3eeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3eeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 -- 172.21.15.33:0/1585795727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 msgr2=0x55cc1b3e3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 -- 172.21.15.33:0/1585795727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1b342e70 con 0x55cc1b3e39b0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26deffd640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26df7fe640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 0x7f26cc0a3a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005fd0 tx=0x7f26d800ced0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.145+0000 7f26de7fc640 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26c8005ab0 tx=0x7f26c8004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7d663e9394560c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26d5ffb640 1 -- 172.21.15.33:0/1585795727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80049c0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26d5ffb640 1 -- 172.21.15.33:0/1585795727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c8004040 con 0x55cc1b3e39b0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26d5ffb640 1 -- 172.21.15.33:0/1585795727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80053c0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 -- 172.21.15.33:0/1585795727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 msgr2=0x55cc1b3ec240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f26c8005ab0 tx=0x7f26c8004e50 comp rx=0 tx=0).stop 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 -- 172.21.15.33:0/1585795727 shutdown_connections 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3eeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3ec240 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 --2- 172.21.15.33:0/1585795727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 -- 172.21.15.33:0/1585795727 >> 172.21.15.33:0/1585795727 conn(0x55cc1b3dd9b0 msgr2=0x55cc1b3ddd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 -- 172.21.15.33:0/1585795727 shutdown_connections 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.146+0000 7f26e8616740 1 -- 172.21.15.33:0/1585795727 wait complete. 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 -- start start 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 Processor -- start 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3041e0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b303e80 con 0x55cc1b3e3090 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b2c90c0 con 0x55cc1b3ec780 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26e8616740 10 client.0 _ll_register_callbacks cb 0x55cc1b2eeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26de7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26de7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54056/0 (socket says 172.21.15.33:54056) 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26de7fc640 1 -- 172.21.15.33:0/885778526 learned_addr learned my addr 172.21.15.33:0/885778526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.147+0000 7f26df7fe640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26deffd640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 -- 172.21.15.33:0/885778526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 msgr2=0x55cc1b3ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 -- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 msgr2=0x55cc1b3e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 -- 172.21.15.33:0/885778526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e010d3a0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26deffd640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26de7fc640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f26c802c170 tx=0x7f26c80043e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dafaf8e7ba01163 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26beffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c802cb90 con 0x55cc1b3e39b0 2024-01-25T16:12:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f26c80051a0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc1b344040 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26c8005380 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26e8616740 10 client.15810 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cc1b3496d0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.148+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26c805d080 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f26c803c040 con 0x55cc1b3e39b0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 10 client.15810 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 20 client.15810 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b3fbfb0 0x55cc1b3fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 4 client.15810 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc1b343d60 con 0x55cc1b3fbfb0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26deffd640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b3fbfb0 0x55cc1b3fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26deffd640 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b3fbfb0 0x55cc1b3fe3c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ee5de7faf8399b server_cookie=ca8156ddb8d9d4bc in_seq=0 out_seq=0 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.149+0000 7f26beffd640 10 client.15810 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26beffd640 1 -- 172.21.15.33:0/885778526 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc1b343d60 con 0x55cc1b3fbfb0 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26beffd640 10 client.15810 handle_command_reply: tid=0 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 client.15810 shutdown 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 msgr2=0x55cc1b3e9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f26c802c170 tx=0x7f26c80043e0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 shutdown_connections 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b3fbfb0 0x55cc1b3fe3c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26beffd640 0 client.15810 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9ba0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 --2- 172.21.15.33:0/885778526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 >> 172.21.15.33:0/885778526 conn(0x55cc1b3dd9b0 msgr2=0x55cc1b3df700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 shutdown_connections 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 1 -- 172.21.15.33:0/885778526 wait complete. 2024-01-25T16:12:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.150+0000 7f26e8616740 20 client.15810 trim_cache size 0 max 16384 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 Processor -- start 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 -- start start 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3e8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3de9b0 con 0x55cc1b3e3090 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3d9270 con 0x55cc1b3e39b0 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26e8616740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3d7e60 con 0x55cc1b3ec780 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54072/0 (socket says 172.21.15.33:54072) 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 -- 172.21.15.33:0/472474485 learned_addr learned my addr 172.21.15.33:0/472474485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26df7fe640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3e8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26de7fc640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 -- 172.21.15.33:0/472474485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 msgr2=0x55cc1b3e8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3e8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 -- 172.21.15.33:0/472474485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 msgr2=0x55cc1b3e9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 -- 172.21.15.33:0/472474485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc1b2e6060 con 0x55cc1b3e3090 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26de7fc640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.164+0000 7f26deffd640 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26d0099cb0 tx=0x7f26d00cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fee77af422fd6d3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26d57fa640 1 -- 172.21.15.33:0/472474485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d0098350 con 0x55cc1b3e3090 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26d57fa640 1 -- 172.21.15.33:0/472474485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d0005470 con 0x55cc1b3e3090 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26d57fa640 1 -- 172.21.15.33:0/472474485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d00a1eb0 con 0x55cc1b3e3090 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26e8616740 1 -- 172.21.15.33:0/472474485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 msgr2=0x55cc1b3ebd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26e8616740 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f26d0099cb0 tx=0x7f26d00cc040 comp rx=0 tx=0).stop 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26e8616740 1 -- 172.21.15.33:0/472474485 shutdown_connections 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.165+0000 7f26e8616740 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3ec780 0x55cc1b3e8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3e9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 --2- 172.21.15.33:0/472474485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3e3090 0x55cc1b3ebd90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- 172.21.15.33:0/472474485 >> 172.21.15.33:0/472474485 conn(0x55cc1b3dd9b0 msgr2=0x55cc1b3e7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- 172.21.15.33:0/472474485 shutdown_connections 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- 172.21.15.33:0/472474485 wait complete. 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- start start 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 Processor -- start 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3f3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3f2a00 con 0x55cc1b3ec780 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b3f2b70 con 0x55cc1b3e3090 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1b4524f0 con 0x55cc1b3e39b0 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.166+0000 7f26e8616740 10 client.0 _ll_register_callbacks cb 0x7f26c8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54080/0 (socket says 172.21.15.33:54080) 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26deffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 -- 172.21.15.33:0/4104494464 learned_addr learned my addr 172.21.15.33:0/4104494464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26de7fc640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 -- 172.21.15.33:0/4104494464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3e39b0 msgr2=0x55cc1b3f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 -- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 msgr2=0x55cc1b3f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 -- 172.21.15.33:0/4104494464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d0078e70 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26df7fe640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26d8012e00 tx=0x7f26d8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41aba081cef9a6d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.167+0000 7f26beffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d8024070 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f26d8019070 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc1b3d90b0 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26d8018030 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26e8616740 10 client.15816 fetch_fsmap learned FSMap version 26 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.168+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55cc1b30ba80 con 0x55cc1b3ec780 2024-01-25T16:12:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d8035080 con 0x55cc1b3ec780 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f26d8010550 con 0x55cc1b3ec780 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 10 client.15816 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 20 client.15816 populate_metadata read hostname 'smithi033' 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b453d40 0x55cc1b456160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 4 client.15816 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f26c8004040 con 0x55cc1b453d40 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26deffd640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b453d40 0x55cc1b456160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26deffd640 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b453d40 0x55cc1b456160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32817423d7dab583 server_cookie=72efeb857ee1233b in_seq=0 out_seq=0 2024-01-25T16:12:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.169+0000 7f26beffd640 10 client.15816 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26beffd640 1 -- 172.21.15.33:0/4104494464 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f26c8004040 con 0x55cc1b453d40 2024-01-25T16:12:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26beffd640 10 client.15816 handle_command_reply: tid=0 2024-01-25T16:12:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 client.15816 shutdown 2024-01-25T16:12:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 msgr2=0x55cc1b3f22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26d8012e00 tx=0x7f26d8004730 comp rx=0 tx=0).stop 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 shutdown_connections 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55cc1b453d40 0x55cc1b456160 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cc1b3ec780 0x55cc1b3f22b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55cc1b3e39b0 0x55cc1b3f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 --2- 172.21.15.33:0/4104494464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cc1b3e3090 0x55cc1b3f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 >> 172.21.15.33:0/4104494464 conn(0x55cc1b3dd9b0 msgr2=0x55cc1b3dee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26beffd640 0 client.15816 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 shutdown_connections 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 1 -- 172.21.15.33:0/4104494464 wait complete. 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.206+0000 7f26e8616740 20 client.15816 trim_cache size 0 max 16384 2024-01-25T16:12:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:12:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:12:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:12:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:12:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 msgr2=0x7f26cc0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 0x7f26cc0a3a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f26d8005fd0 tx=0x7f26d800ced0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 msgr2=0x7f26e0199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f26d000ea80 tx=0x7f26d000ef40 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 shutdown_connections 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f26cc0a1570 0x7f26cc0a3a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26e0104b70 0x7f26e019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26e01007e0 0x7f26e0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 --2- 172.21.15.33:0/2368375062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26e00ffe40 0x7f26e0199450 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 >> 172.21.15.33:0/2368375062 conn(0x7f26e00769c0 msgr2=0x7f26e010b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 shutdown_connections 2024-01-25T16:12:55.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:12:55.207+0000 7f26e597d640 1 -- 172.21.15.33:0/2368375062 wait complete. 2024-01-25T16:12:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:55 smithi033 ceph-mon[31417]: pgmap v156: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:55 smithi152 ceph-mon[36015]: pgmap v156: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:55 smithi076 ceph-mon[35642]: pgmap v156: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 761 KiB/s wr, 127 op/s 2024-01-25T16:12:55.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 102977 2.15 MB/sec execute 112 sec latency 4874.803 ms 2024-01-25T16:12:56.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 105340 2.20 MB/sec execute 113 sec latency 18.543 ms 2024-01-25T16:12:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:57 smithi033 ceph-mon[31417]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 743 KiB/s wr, 138 op/s 2024-01-25T16:12:57.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:57 smithi152 ceph-mon[36015]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 743 KiB/s wr, 138 op/s 2024-01-25T16:12:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:57 smithi076 ceph-mon[35642]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 743 KiB/s wr, 138 op/s 2024-01-25T16:12:57.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 105340 2.18 MB/sec execute 114 sec latency 1018.659 ms 2024-01-25T16:12:58.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 105340 2.16 MB/sec execute 115 sec latency 2018.805 ms 2024-01-25T16:12:59.471 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:59 smithi076 ceph-mon[35642]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:12:59.471 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:12:59 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:12:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:59 smithi033 ceph-mon[31417]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:12:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:12:59 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:12:59.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:59 smithi152 ceph-mon[36015]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:12:59.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:12:59 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:12:59.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 105340 2.15 MB/sec execute 116 sec latency 3018.850 ms 2024-01-25T16:13:00.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:00 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:13:00.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:00 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:13:00.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:00 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:13:00.636 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:00 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:13:00.636 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:00 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:13:00.636 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:00 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:13:00.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:00 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:13:00.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:00 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:13:00.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:00 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:13:00.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 106258 2.19 MB/sec execute 117 sec latency 3679.258 ms 2024-01-25T16:13:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:01 smithi033 ceph-mon[31417]: pgmap v159: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:13:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:01 smithi152 ceph-mon[36015]: pgmap v159: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:13:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:01 smithi076 ceph-mon[35642]: pgmap v159: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 685 KiB/s rd, 271 KiB/s wr, 53 op/s 2024-01-25T16:13:01.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 107484 2.23 MB/sec execute 118 sec latency 23.058 ms 2024-01-25T16:13:02.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 110253 2.35 MB/sec execute 119 sec latency 21.828 ms 2024-01-25T16:13:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:03 smithi033 ceph-mon[31417]: pgmap v160: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 557 KiB/s wr, 96 op/s 2024-01-25T16:13:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:03 smithi152 ceph-mon[36015]: pgmap v160: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 557 KiB/s wr, 96 op/s 2024-01-25T16:13:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:03 smithi076 ceph-mon[35642]: pgmap v160: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 557 KiB/s wr, 96 op/s 2024-01-25T16:13:03.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 112767 2.43 MB/sec execute 120 sec latency 14.171 ms 2024-01-25T16:13:04.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 114802 2.49 MB/sec execute 121 sec latency 18.277 ms 2024-01-25T16:13:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:05 smithi033 ceph-mon[31417]: pgmap v161: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 394 KiB/s wr, 74 op/s 2024-01-25T16:13:05.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:05 smithi152 ceph-mon[36015]: pgmap v161: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 394 KiB/s wr, 74 op/s 2024-01-25T16:13:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:05 smithi076 ceph-mon[35642]: pgmap v161: 97 pgs: 97 active+clean; 43 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 394 KiB/s wr, 74 op/s 2024-01-25T16:13:05.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 117458 2.61 MB/sec execute 122 sec latency 19.916 ms 2024-01-25T16:13:06.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 120082 2.69 MB/sec execute 123 sec latency 16.045 ms 2024-01-25T16:13:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:07 smithi033 ceph-mon[31417]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-01-25T16:13:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:07 smithi152 ceph-mon[36015]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-01-25T16:13:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:07 smithi076 ceph-mon[35642]: pgmap v162: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-01-25T16:13:07.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.72 MB/sec execute 124 sec latency 385.957 ms 2024-01-25T16:13:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 -- 172.21.15.33:0/1103145616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 msgr2=0x7f386410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 --2- 172.21.15.33:0/1103145616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386410f920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f385c009ca0 tx=0x7f385c02c180 comp rx=0 tx=0).stop 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 -- 172.21.15.33:0/1103145616 shutdown_connections 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 --2- 172.21.15.33:0/1103145616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386410f920 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 --2- 172.21.15.33:0/1103145616 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864069a30 0x7f3864069e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 --2- 172.21.15.33:0/1103145616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864101c50 0x7f38640694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.303+0000 7f386be23640 1 -- 172.21.15.33:0/1103145616 >> 172.21.15.33:0/1103145616 conn(0x7f38640769c0 msgr2=0x7f3864076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- 172.21.15.33:0/1103145616 shutdown_connections 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- 172.21.15.33:0/1103145616 wait complete. 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 Processor -- start 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- start start 2024-01-25T16:13:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864069a30 0x7f38641992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 0x7f38641997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386419e160 con 0x7f386410d4b0 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386419e2d0 con 0x7f3864069a30 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386be23640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386419e490 con 0x7f3864101c50 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386a3c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.304+0000 7f386a3c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44468/0 (socket says 172.21.15.33:44468) 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 -- 172.21.15.33:0/2302288165 learned_addr learned my addr 172.21.15.33:0/2302288165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f38693be640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 0x7f38641997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f3869bbf640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864069a30 0x7f38641992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 -- 172.21.15.33:0/2302288165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 msgr2=0x7f38641997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 0x7f38641997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 -- 172.21.15.33:0/2302288165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864069a30 msgr2=0x7f38641992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864069a30 0x7f38641992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 -- 172.21.15.33:0/2302288165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c009900 con 0x7f386410d4b0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f38693be640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 0x7f38641997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386a3c0640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f385c02c790 tx=0x7f385c034830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f385affd640 1 -- 172.21.15.33:0/2302288165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c003d80 con 0x7f386410d4b0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f386419e710 con 0x7f386410d4b0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f385affd640 1 -- 172.21.15.33:0/2302288165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f385c032b70 con 0x7f386410d4b0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f385affd640 1 -- 172.21.15.33:0/2302288165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c004780 con 0x7f386410d4b0 2024-01-25T16:13:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.305+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386419e9e0 con 0x7f386410d4b0 2024-01-25T16:13:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.307+0000 7f385affd640 1 -- 172.21.15.33:0/2302288165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f385c00a040 con 0x7f386410d4b0 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.308+0000 7f386e9df740 1 Processor -- start 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.308+0000 7f386e9df740 1 -- start start 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f385affd640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 0x7f38500a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a900070 con 0x556b4a991090 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a8fd760 con 0x556b4a9919b0 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386e9df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a8f9050 con 0x556b4a9a2f90 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f38693be640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 0x7f38500a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f385affd640 1 -- 172.21.15.33:0/2302288165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f385c0efad0 con 0x7f386410d4b0 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44474/0 (socket says 172.21.15.33:44474) 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 -- 172.21.15.33:0/2106343936 learned_addr learned my addr 172.21.15.33:0/2106343936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f38693be640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386a3c0640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 -- 172.21.15.33:0/2106343936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 msgr2=0x556b4a9a5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 -- 172.21.15.33:0/2106343936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 msgr2=0x556b4a9a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 -- 172.21.15.33:0/2106343936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4a8f0e70 con 0x556b4a991090 2024-01-25T16:13:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f386a3c0640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f38693be640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 0x7f38500a39b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f386410dcb0 tx=0x7f38540138e0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.309+0000 7f3869bbf640 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3860009cf0 tx=0x7f3860011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a057a93434ef36b1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f384ffff640 1 -- 172.21.15.33:0/2106343936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860009ea0 con 0x556b4a991090 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f384ffff640 1 -- 172.21.15.33:0/2106343936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3860004230 con 0x556b4a991090 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f384ffff640 1 -- 172.21.15.33:0/2106343936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860019750 con 0x556b4a991090 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 -- 172.21.15.33:0/2106343936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 msgr2=0x556b4a991470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3860009cf0 tx=0x7f3860011810 comp rx=0 tx=0).stop 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 -- 172.21.15.33:0/2106343936 shutdown_connections 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 --2- 172.21.15.33:0/2106343936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a991470 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.310+0000 7f386e9df740 1 -- 172.21.15.33:0/2106343936 >> 172.21.15.33:0/2106343936 conn(0x556b4a98b9b0 msgr2=0x556b4a98bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- 172.21.15.33:0/2106343936 shutdown_connections 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- 172.21.15.33:0/2106343936 wait complete. 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- start start 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 Processor -- start 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a991090 0x556b4a99feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a8b21e0 con 0x556b4a9919b0 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a8b1e80 con 0x556b4a991090 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a8770c0 con 0x556b4a9a2f90 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.311+0000 7f386e9df740 10 client.0 _ll_register_callbacks cb 0x556b4a89cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44484/0 (socket says 172.21.15.33:44484) 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 -- 172.21.15.33:0/317867268 learned_addr learned my addr 172.21.15.33:0/317867268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f3869bbf640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a991090 0x556b4a99feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f386a3c0640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 -- 172.21.15.33:0/317867268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 msgr2=0x556b4a9a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 -- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a991090 msgr2=0x556b4a99feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a991090 0x556b4a99feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 -- 172.21.15.33:0/317867268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3860009950 con 0x556b4a9919b0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f386a3c0640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f38693be640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3854011760 tx=0x7f3854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d171b9eabd066642 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f384d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e760 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f385400e8c0 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b4a8f2040 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f385400ea80 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f386e9df740 10 client.15828 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.312+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556b4a8f76d0 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b4a8f2040 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3854020cc0 con 0x556b4a9919b0 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 10 client.15828 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 20 client.15828 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4a9b2d00 0x556b4a9b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 4 client.15828 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b4a8f1d60 con 0x556b4a9b2d00 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.313+0000 7f3869bbf640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4a9b2d00 0x556b4a9b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f384d7fa640 10 client.15828 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f3869bbf640 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4a9b2d00 0x556b4a9b5120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e501a6e90ae73764 server_cookie=14aee525f9d105da in_seq=0 out_seq=0 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f384d7fa640 1 -- 172.21.15.33:0/317867268 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556b4a8f1d60 con 0x556b4a9b2d00 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f384d7fa640 10 client.15828 handle_command_reply: tid=0 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f386e9df740 1 client.15828 shutdown 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 msgr2=0x556b4a9a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.314+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3854011760 tx=0x7f3854004730 comp rx=0 tx=0).stop 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 shutdown_connections 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4a9b2d00 0x556b4a9b5120 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a9919b0 0x556b4a9a03f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 --2- 172.21.15.33:0/317867268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a991090 0x556b4a99feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f384d7fa640 0 client.15828 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 >> 172.21.15.33:0/317867268 conn(0x556b4a98b9b0 msgr2=0x556b4a98d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 shutdown_connections 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 1 -- 172.21.15.33:0/317867268 wait complete. 2024-01-25T16:13:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.315+0000 7f386e9df740 20 client.15828 trim_cache size 0 max 16384 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 Processor -- start 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 -- start start 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a990f10 con 0x556b4a991090 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a9a5740 con 0x556b4a9919b0 2024-01-25T16:13:08.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f386e9df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a992040 con 0x556b4a9a2f90 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44500/0 (socket says 172.21.15.33:44500) 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f3869bbf640 1 -- 172.21.15.33:0/1496344400 learned_addr learned my addr 172.21.15.33:0/1496344400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.328+0000 7f38693be640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f386a3c0640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 -- 172.21.15.33:0/1496344400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 msgr2=0x556b4a9a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 -- 172.21.15.33:0/1496344400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 msgr2=0x556b4a9a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 -- 172.21.15.33:0/1496344400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b4a894060 con 0x556b4a991090 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f386a3c0640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:13:08.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f3869bbf640 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3860003e20 tx=0x7f3860003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d23588422c5fe server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f384f7fe640 1 -- 172.21.15.33:0/1496344400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860029070 con 0x556b4a991090 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f384f7fe640 1 -- 172.21.15.33:0/1496344400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f386002e020 con 0x556b4a991090 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f384f7fe640 1 -- 172.21.15.33:0/1496344400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38600054a0 con 0x556b4a991090 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f386e9df740 1 -- 172.21.15.33:0/1496344400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 msgr2=0x556b4a99f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.329+0000 7f386e9df740 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3860003e20 tx=0x7f3860003e80 comp rx=0 tx=0).stop 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- 172.21.15.33:0/1496344400 shutdown_connections 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4a9a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- 172.21.15.33:0/1496344400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a99f010 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- 172.21.15.33:0/1496344400 >> 172.21.15.33:0/1496344400 conn(0x556b4a98b9b0 msgr2=0x556b4a98a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:08.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- 172.21.15.33:0/1496344400 shutdown_connections 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- 172.21.15.33:0/1496344400 wait complete. 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- start start 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 Processor -- start 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4aa09210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4a9a9ae0 con 0x556b4a991090 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4aa09750 con 0x556b4a9919b0 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b4aa098c0 con 0x556b4a9a2f90 2024-01-25T16:13:08.339 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f386e9df740 10 client.0 _ll_register_callbacks cb 0x7f38600097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.330+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44504/0 (socket says 172.21.15.33:44504) 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 -- 172.21.15.33:0/306346808 learned_addr learned my addr 172.21.15.33:0/306346808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f38693be640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f386a3c0640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4aa09210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 -- 172.21.15.33:0/306346808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 msgr2=0x556b4aa09210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4aa09210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 -- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 msgr2=0x556b4a9a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 -- 172.21.15.33:0/306346808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b4a8f0e70 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f38693be640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f386a3c0640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4aa09210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f3869bbf640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3860003e50 tx=0x7f3860003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2572cd8c9318e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f384d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3860029040 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f386002f030 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3860004230 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3860005430 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f386e9df740 10 client.15834 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.331+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x556b4a8b9a80 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3860004230 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3860021040 con 0x556b4a991090 2024-01-25T16:13:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 10 client.15834 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 20 client.15834 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4aa0ac90 0x556b4aa0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 4 client.15834 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556b4a5f8600 con 0x556b4aa0ac90 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.332+0000 7f38693be640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4aa0ac90 0x556b4aa0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.333+0000 7f38693be640 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4aa0ac90 0x556b4aa0d080 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee30c06f4c947531 server_cookie=585c75765ad6f8fd in_seq=0 out_seq=0 2024-01-25T16:13:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:08.333+0000 7f384d7fa640 10 client.15834 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:08.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:08 smithi033 ceph-mon[31417]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:08.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:08 smithi152 ceph-mon[36015]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:08.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:08 smithi076 ceph-mon[35642]: pgmap v163: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:08.772 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.70 MB/sec execute 125 sec latency 1385.986 ms 2024-01-25T16:13:09.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.68 MB/sec execute 126 sec latency 2386.069 ms 2024-01-25T16:13:10.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.66 MB/sec execute 127 sec latency 3386.211 ms 2024-01-25T16:13:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:11 smithi033 ceph-mon[31417]: pgmap v164: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:11 smithi152 ceph-mon[36015]: pgmap v164: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:11 smithi076 ceph-mon[35642]: pgmap v164: 97 pgs: 97 active+clean; 55 MiB data, 711 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-01-25T16:13:11.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.64 MB/sec execute 128 sec latency 4386.353 ms 2024-01-25T16:13:12.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.62 MB/sec execute 129 sec latency 5386.450 ms 2024-01-25T16:13:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:13 smithi033 ceph-mon[31417]: pgmap v165: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-01-25T16:13:13.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:13 smithi152 ceph-mon[36015]: pgmap v165: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-01-25T16:13:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:13 smithi076 ceph-mon[35642]: pgmap v165: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 245 op/s 2024-01-25T16:13:13.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.60 MB/sec execute 130 sec latency 6386.565 ms 2024-01-25T16:13:14.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121227 2.58 MB/sec execute 131 sec latency 7386.657 ms 2024-01-25T16:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f384d7fa640 1 -- 172.21.15.33:0/306346808 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556b4a5f8600 con 0x556b4aa0ac90 2024-01-25T16:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f384d7fa640 10 client.15834 handle_command_reply: tid=0 2024-01-25T16:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 client.15834 shutdown 2024-01-25T16:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 msgr2=0x556b4a9a8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:15.486 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3860003e50 tx=0x7f3860003ef0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 shutdown_connections 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x556b4aa0ac90 0x556b4aa0d080 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x556b4a9a2f90 0x556b4aa09210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556b4a9919b0 0x556b4a9a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f384d7fa640 0 client.15834 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 --2- 172.21.15.33:0/306346808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b4a991090 0x556b4a9a8e80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 >> 172.21.15.33:0/306346808 conn(0x556b4a98b9b0 msgr2=0x556b4a98aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 shutdown_connections 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.483+0000 7f386e9df740 1 -- 172.21.15.33:0/306346808 wait complete. 2024-01-25T16:13:15.487 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386e9df740 20 client.15834 trim_cache size 0 max 16384 2024-01-25T16:13:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:13:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:13:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:13:15.488 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 msgr2=0x7f38500a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 0x7f38500a39b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f386410dcb0 tx=0x7f38540138e0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 msgr2=0x7f386419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f385c02c790 tx=0x7f385c034830 comp rx=0 tx=0).stop 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 shutdown_connections 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f38500a14f0 0x7f38500a39b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f386410d4b0 0x7f386419db60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.489 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3864101c50 0x7f38641997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 --2- 172.21.15.33:0/2302288165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3864069a30 0x7f38641992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.484+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 >> 172.21.15.33:0/2302288165 conn(0x7f38640769c0 msgr2=0x7f38640fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.485+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 shutdown_connections 2024-01-25T16:13:15.490 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:15.485+0000 7f386be23640 1 -- 172.21.15.33:0/2302288165 wait complete. 2024-01-25T16:13:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:15 smithi033 ceph-mon[31417]: pgmap v166: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-01-25T16:13:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:15 smithi152 ceph-mon[36015]: pgmap v166: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-01-25T16:13:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:15 smithi076 ceph-mon[35642]: pgmap v166: 97 pgs: 97 active+clean; 56 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-01-25T16:13:15.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121743 2.57 MB/sec execute 132 sec latency 8118.579 ms 2024-01-25T16:13:16.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 124288 2.67 MB/sec execute 133 sec latency 15.295 ms 2024-01-25T16:13:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:17 smithi033 ceph-mon[31417]: pgmap v167: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-01-25T16:13:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:17 smithi152 ceph-mon[36015]: pgmap v167: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-01-25T16:13:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:17 smithi076 ceph-mon[35642]: pgmap v167: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.3 MiB/s wr, 236 op/s 2024-01-25T16:13:17.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 125845 2.70 MB/sec execute 134 sec latency 135.844 ms 2024-01-25T16:13:18.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 128142 2.78 MB/sec execute 135 sec latency 19.772 ms 2024-01-25T16:13:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:19 smithi033 ceph-mon[31417]: pgmap v168: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:19.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:19 smithi152 ceph-mon[36015]: pgmap v168: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:19 smithi076 ceph-mon[35642]: pgmap v168: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:19.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 130488 2.84 MB/sec execute 136 sec latency 15.549 ms 2024-01-25T16:13:20.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 132446 2.90 MB/sec execute 137 sec latency 19.789 ms 2024-01-25T16:13:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:21 smithi033 ceph-mon[31417]: pgmap v169: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:21 smithi152 ceph-mon[36015]: pgmap v169: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:21 smithi076 ceph-mon[35642]: pgmap v169: 97 pgs: 97 active+clean; 49 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-01-25T16:13:21.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 135054 3.00 MB/sec execute 138 sec latency 17.767 ms 2024-01-25T16:13:22.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 136912 3.02 MB/sec execute 139 sec latency 49.611 ms 2024-01-25T16:13:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:23 smithi033 ceph-mon[31417]: pgmap v170: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-01-25T16:13:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:23 smithi152 ceph-mon[36015]: pgmap v170: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-01-25T16:13:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:23 smithi076 ceph-mon[35642]: pgmap v170: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.1 MiB/s wr, 215 op/s 2024-01-25T16:13:23.773 INFO:tasks.workunit.client.0.smithi033.stdout: 1 138961 3.09 MB/sec execute 140 sec latency 36.447 ms 2024-01-25T16:13:24.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 141350 3.16 MB/sec execute 141 sec latency 16.174 ms 2024-01-25T16:13:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:25 smithi033 ceph-mon[31417]: pgmap v171: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-01-25T16:13:25.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:25 smithi152 ceph-mon[36015]: pgmap v171: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-01-25T16:13:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:25 smithi076 ceph-mon[35642]: pgmap v171: 97 pgs: 97 active+clean; 55 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 125 op/s 2024-01-25T16:13:25.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 143364 3.22 MB/sec execute 142 sec latency 17.027 ms 2024-01-25T16:13:26.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 145749 3.29 MB/sec execute 143 sec latency 26.449 ms 2024-01-25T16:13:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:27 smithi033 ceph-mon[31417]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-01-25T16:13:27.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:27 smithi152 ceph-mon[36015]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-01-25T16:13:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:27 smithi076 ceph-mon[35642]: pgmap v172: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-01-25T16:13:27.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 147838 3.32 MB/sec execute 144 sec latency 17.625 ms 2024-01-25T16:13:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 -- 172.21.15.33:0/3245638924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4100800 msgr2=0x7fb8b410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 --2- 172.21.15.33:0/3245638924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4100800 0x7fb8b410ca20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009ca0 tx=0x7fb8ac02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 -- 172.21.15.33:0/3245638924 shutdown_connections 2024-01-25T16:13:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 --2- 172.21.15.33:0/3245638924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4100800 0x7fb8b410ca20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.579 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 --2- 172.21.15.33:0/3245638924 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 0x7fb8b41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 --2- 172.21.15.33:0/3245638924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4104b90 0x7fb8b4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 -- 172.21.15.33:0/3245638924 >> 172.21.15.33:0/3245638924 conn(0x7fb8b4076a40 msgr2=0x7fb8b4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 -- 172.21.15.33:0/3245638924 shutdown_connections 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.576+0000 7fb8ba333640 1 -- 172.21.15.33:0/3245638924 wait complete. 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 Processor -- start 2024-01-25T16:13:28.580 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 -- start start 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 0x7fb8b41971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 0x7fb8b41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b419c060 con 0x7fb8b4104b90 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b419c1d0 con 0x7fb8b4100800 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8ba333640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b419c390 con 0x7fb8b40ffe40 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38074/0 (socket says 172.21.15.33:38074) 2024-01-25T16:13:28.582 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 -- 172.21.15.33:0/2827045934 learned_addr learned my addr 172.21.15.33:0/2827045934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 0x7fb8b41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 0x7fb8b41971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 -- 172.21.15.33:0/2827045934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 msgr2=0x7fb8b41971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 0x7fb8b41971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.577+0000 7fb8b88d0640 1 -- 172.21.15.33:0/2827045934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 msgr2=0x7fb8b41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b88d0640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 0x7fb8b41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b88d0640 1 -- 172.21.15.33:0/2827045934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac009900 con 0x7fb8b4104b90 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 0x7fb8b41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b88d0640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac02c700 tx=0x7fb8ac032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b17fa640 1 -- 172.21.15.33:0/2827045934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac032800 con 0x7fb8b4104b90 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b419c610 con 0x7fb8b4104b90 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b17fa640 1 -- 172.21.15.33:0/2827045934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8ac003e30 con 0x7fb8b4104b90 2024-01-25T16:13:28.583 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8b17fa640 1 -- 172.21.15.33:0/2827045934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0047f0 con 0x7fb8b4104b90 2024-01-25T16:13:28.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.578+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b419c8e0 con 0x7fb8b4104b90 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 Processor -- start 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b17fa640 1 -- 172.21.15.33:0/2827045934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb8ac03b040 con 0x7fb8b4104b90 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 -- start start 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d0cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d1a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c7c070 con 0x55b1e8d1ab90 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c79760 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c75050 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.591 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38084/0 (socket says 172.21.15.33:38084) 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b37fe640 1 -- 172.21.15.33:0/3618248457 learned_addr learned my addr 172.21.15.33:0/3618248457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b17fa640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 0x7fb89c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d0cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.581+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d1a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b88d0640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 0x7fb89c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 -- 172.21.15.33:0/3618248457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 msgr2=0x55b1e8d1a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d1a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 -- 172.21.15.33:0/3618248457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 msgr2=0x55b1e8d0cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d0cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 -- 172.21.15.33:0/3618248457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e8c6ce70 con 0x55b1e8d1ab90 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b17fa640 1 -- 172.21.15.33:0/2827045934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb8ac0ef710 con 0x7fb8b4104b90 2024-01-25T16:13:28.592 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8012b80 tx=0x7fb8a800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b59bf2df4c67d10 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb89a7fc640 1 -- 172.21.15.33:0/3618248457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8012c70 con 0x55b1e8d1ab90 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb89a7fc640 1 -- 172.21.15.33:0/3618248457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a80044f0 con 0x55b1e8d1ab90 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.582+0000 7fb89a7fc640 1 -- 172.21.15.33:0/3618248457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8003a10 con 0x55b1e8d1ab90 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/3618248457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 msgr2=0x55b1e8d1cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8b88d0640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 0x7fb89c0a39b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0bc310 tx=0x7fb8ac04de10 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8012b80 tx=0x7fb8a800bb90 comp rx=0 tx=0).stop 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/3618248457 shutdown_connections 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d1cf80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d1a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/3618248457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d0cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/3618248457 >> 172.21.15.33:0/3618248457 conn(0x55b1e8d079b0 msgr2=0x55b1e8d07d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/3618248457 shutdown_connections 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/3618248457 wait complete. 2024-01-25T16:13:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 -- start start 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.583+0000 7fb8bcfcd740 1 Processor -- start 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d17b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c2e1e0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8c2de80 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8bf30c0 con 0x55b1e8d1ab90 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 10 client.0 _ll_register_callbacks cb 0x55b1e8c18ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38088/0 (socket says 172.21.15.33:38088) 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 -- 172.21.15.33:0/1335640735 learned_addr learned my addr 172.21.15.33:0/1335640735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d17b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 -- 172.21.15.33:0/1335640735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 msgr2=0x55b1e8d185a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d185a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 -- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 msgr2=0x55b1e8d17b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d17b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 -- 172.21.15.33:0/1335640735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a801a040 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4012770 tx=0x7fb8a4012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb622aaaa66a848 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4017070 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb8a400f920 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.584+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e8c6e040 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a400fae0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 10 client.15846 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b1e8c736d0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a4028080 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8a4010cc0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 20 client.15846 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d2aa20 0x55b1e8d2ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 4 client.15846 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1e8c6dd60 con 0x55b1e8d2aa20 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.585+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d2aa20 0x55b1e8d2ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.586+0000 7fb8937fe640 10 client.15846 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.586+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d2aa20 0x55b1e8d2ce40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738356d4996795a8 server_cookie=f1342cb9ebfbe551 in_seq=0 out_seq=0 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.586+0000 7fb8937fe640 1 -- 172.21.15.33:0/1335640735 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1e8c6dd60 con 0x55b1e8d2aa20 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.586+0000 7fb8937fe640 10 client.15846 handle_command_reply: tid=0 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.586+0000 7fb8bcfcd740 1 client.15846 shutdown 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 msgr2=0x55b1e8d18060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4012770 tx=0x7fb8a4012c30 comp rx=0 tx=0).stop 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 shutdown_connections 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d2aa20 0x55b1e8d2ce40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d18060 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8937fe640 0 client.15846 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1335640735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d17b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 >> 172.21.15.33:0/1335640735 conn(0x55b1e8d079b0 msgr2=0x55b1e8d09700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 shutdown_connections 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1335640735 wait complete. 2024-01-25T16:13:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.587+0000 7fb8bcfcd740 20 client.15846 trim_cache size 0 max 16384 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 Processor -- start 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 -- start start 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d09510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d071d0 con 0x55b1e8d1ab90 2024-01-25T16:13:28.608 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d032e0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d27480 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38100/0 (socket says 172.21.15.33:38100) 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1373680931 learned_addr learned my addr 172.21.15.33:0/1373680931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d09510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1373680931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0cbb0 msgr2=0x55b1e8d183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d183c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1373680931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0d4d0 msgr2=0x55b1e8d09510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d09510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1373680931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e8c10060 con 0x55b1e8d1ab90 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8012b50 tx=0x7fb8a800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cedde794d9a994c1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.601+0000 7fb899ffb640 1 -- 172.21.15.33:0/1373680931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802d480 con 0x55b1e8d1ab90 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb899ffb640 1 -- 172.21.15.33:0/1373680931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a8033070 con 0x55b1e8d1ab90 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb899ffb640 1 -- 172.21.15.33:0/1373680931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8004660 con 0x55b1e8d1ab90 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1373680931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 msgr2=0x55b1e8d21850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8012b50 tx=0x7fb8a800b390 comp rx=0 tx=0).stop 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1373680931 shutdown_connections 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d21850 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d09510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1373680931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1373680931 >> 172.21.15.33:0/1373680931 conn(0x55b1e8d079b0 msgr2=0x55b1e8d08e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1373680931 shutdown_connections 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1373680931 wait complete. 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.602+0000 7fb8bcfcd740 1 -- start start 2024-01-25T16:13:28.610 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 Processor -- start 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d7af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d7b4d0 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d7b610 con 0x55b1e8d1ab90 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e8d7b7d0 con 0x55b1e8d0d4d0 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8bcfcd740 10 client.0 _ll_register_callbacks cb 0x7fb8a800bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38110/0 (socket says 172.21.15.33:38110) 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1944277982 learned_addr learned my addr 172.21.15.33:0/1944277982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d7af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1944277982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 msgr2=0x55b1e8d21210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d21210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d1ab90 msgr2=0x55b1e8d7af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d7af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.603+0000 7fb8b37fe640 1 -- 172.21.15.33:0/1944277982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a8026040 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8b37fe640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8010df0 tx=0x7fb8a800bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d0091c78ce18b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802d670 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb8a8000e40 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8a80044f0 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a803b470 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:28.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b1e8c35a80 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8a80044f0 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb8a80393c0 con 0x55b1e8d0cbb0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 20 client.15852 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d82cd0 0x55b1e8d850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 4 client.15852 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.604+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b1e8974600 con 0x55b1e8d82cd0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.605+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d82cd0 0x55b1e8d850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.605+0000 7fb8b3fff640 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d82cd0 0x55b1e8d850c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a35d60540151b0a server_cookie=8670582f599be007 in_seq=0 out_seq=0 2024-01-25T16:13:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.605+0000 7fb8937fe640 10 client.15852 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8937fe640 1 -- 172.21.15.33:0/1944277982 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b1e8974600 con 0x55b1e8d82cd0 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8937fe640 10 client.15852 handle_command_reply: tid=0 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8bcfcd740 1 client.15852 shutdown 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 msgr2=0x55b1e8d20cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8010df0 tx=0x7fb8a800bd20 comp rx=0 tx=0).stop 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 shutdown_connections 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.748+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b1e8d82cd0 0x55b1e8d850c0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b1e8d1ab90 0x55b1e8d7af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b1e8d0d4d0 0x55b1e8d21210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 --2- 172.21.15.33:0/1944277982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1e8d0cbb0 0x55b1e8d20cd0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8937fe640 0 client.15852 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 >> 172.21.15.33:0/1944277982 conn(0x55b1e8d079b0 msgr2=0x55b1e8d06a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 shutdown_connections 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 1 -- 172.21.15.33:0/1944277982 wait complete. 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8bcfcd740 20 client.15852 trim_cache size 0 max 16384 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:13:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 msgr2=0x7fb89c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 0x7fb89c0a39b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac0bc310 tx=0x7fb8ac04de10 comp rx=0 tx=0).stop 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 msgr2=0x7fb8b419ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac02c700 tx=0x7fb8ac032fc0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 shutdown_connections 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb89c0a14f0 0x7fb89c0a39b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8b4104b90 0x7fb8b419ba60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8b4100800 0x7fb8b41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 --2- 172.21.15.33:0/2827045934 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb8b40ffe40 0x7fb8b41971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 >> 172.21.15.33:0/2827045934 conn(0x7fb8b4076a40 msgr2=0x7fb8b40685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 shutdown_connections 2024-01-25T16:13:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:28.749+0000 7fb8ba333640 1 -- 172.21.15.33:0/2827045934 wait complete. 2024-01-25T16:13:28.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 149711 3.38 MB/sec execute 145 sec latency 68.765 ms 2024-01-25T16:13:29.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:29 smithi076 ceph-mon[35642]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:29 smithi033 ceph-mon[31417]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:29.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:29 smithi152 ceph-mon[36015]: pgmap v173: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:29.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 152341 3.46 MB/sec execute 146 sec latency 23.291 ms 2024-01-25T16:13:30.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 154141 3.50 MB/sec execute 147 sec latency 15.817 ms 2024-01-25T16:13:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:31 smithi033 ceph-mon[31417]: pgmap v174: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:31.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:31 smithi152 ceph-mon[36015]: pgmap v174: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:31 smithi076 ceph-mon[35642]: pgmap v174: 97 pgs: 97 active+clean; 61 MiB data, 823 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-01-25T16:13:31.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 156461 3.57 MB/sec execute 148 sec latency 44.376 ms 2024-01-25T16:13:32.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 158535 3.60 MB/sec execute 149 sec latency 19.313 ms 2024-01-25T16:13:33.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:33 smithi033 ceph-mon[31417]: pgmap v175: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 355 op/s 2024-01-25T16:13:33.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:33 smithi152 ceph-mon[36015]: pgmap v175: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 355 op/s 2024-01-25T16:13:33.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:33 smithi076 ceph-mon[35642]: pgmap v175: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.7 MiB/s wr, 355 op/s 2024-01-25T16:13:33.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 160708 3.66 MB/sec execute 150 sec latency 15.721 ms 2024-01-25T16:13:34.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 162913 3.72 MB/sec execute 151 sec latency 125.466 ms 2024-01-25T16:13:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:35 smithi033 ceph-mon[31417]: pgmap v176: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:35.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:35 smithi152 ceph-mon[36015]: pgmap v176: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:35 smithi076 ceph-mon[35642]: pgmap v176: 97 pgs: 97 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:35.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 164842 3.76 MB/sec execute 152 sec latency 15.667 ms 2024-01-25T16:13:36.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167049 3.83 MB/sec execute 153 sec latency 66.794 ms 2024-01-25T16:13:37.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:37 smithi033 ceph-mon[31417]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:37.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:37 smithi152 ceph-mon[36015]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:37 smithi076 ceph-mon[35642]: pgmap v177: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:37.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 169163 3.85 MB/sec execute 154 sec latency 16.043 ms 2024-01-25T16:13:38.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 171251 3.91 MB/sec execute 155 sec latency 15.280 ms 2024-01-25T16:13:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:39 smithi033 ceph-mon[31417]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:39.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:39 smithi152 ceph-mon[36015]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:39 smithi076 ceph-mon[35642]: pgmap v178: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:39.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 173644 3.97 MB/sec execute 156 sec latency 52.343 ms 2024-01-25T16:13:40.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 175462 4.01 MB/sec execute 157 sec latency 16.414 ms 2024-01-25T16:13:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:41 smithi033 ceph-mon[31417]: pgmap v179: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:41.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:41 smithi152 ceph-mon[36015]: pgmap v179: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:41.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:41 smithi076 ceph-mon[35642]: pgmap v179: 97 pgs: 97 active+clean; 71 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.4 MiB/s wr, 272 op/s 2024-01-25T16:13:41.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 177742 4.07 MB/sec execute 158 sec latency 37.942 ms 2024-01-25T16:13:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 -- 172.21.15.33:0/2823478575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8068cf0 msgr2=0x7f79a810db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/2823478575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8068cf0 0x7f79a810db90 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009930 tx=0x7f79a002bc30 comp rx=0 tx=0).stop 2024-01-25T16:13:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 -- 172.21.15.33:0/2823478575 shutdown_connections 2024-01-25T16:13:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/2823478575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8068cf0 0x7f79a810db90 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/2823478575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 0x7f79a80687b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/2823478575 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8106d60 0x7f79a8107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 -- 172.21.15.33:0/2823478575 >> 172.21.15.33:0/2823478575 conn(0x7f79a8076a40 msgr2=0x7f79a8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 -- 172.21.15.33:0/2823478575 shutdown_connections 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.838+0000 7f79ae1e5640 1 -- 172.21.15.33:0/2823478575 wait complete. 2024-01-25T16:13:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 Processor -- start 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 -- start start 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 0x7f79a81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8068cf0 0x7f79a8197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a819c090 con 0x7f79a8106d60 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a819c200 con 0x7f79a8068370 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79ae1e5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a819c3c0 con 0x7f79a8068cf0 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42442/0 (socket says 172.21.15.33:42442) 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79a7fff640 1 -- 172.21.15.33:0/3855106346 learned_addr learned my addr 172.21.15.33:0/3855106346 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.839+0000 7f79a77fe640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 0x7f79a81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8068cf0 0x7f79a8197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 -- 172.21.15.33:0/3855106346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8068cf0 msgr2=0x7f79a8197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8068cf0 0x7f79a8197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 -- 172.21.15.33:0/3855106346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 msgr2=0x7f79a81971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 0x7f79a81971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 -- 172.21.15.33:0/3855106346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009560 con 0x7f79a8106d60 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f79a0005940 tx=0x7f79a002ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a4ff9640 1 -- 172.21.15.33:0/3855106346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0004dd0 con 0x7f79a8106d60 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a819c640 con 0x7f79a8106d60 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a4ff9640 1 -- 172.21.15.33:0/3855106346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a00054c0 con 0x7f79a8106d60 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79a4ff9640 1 -- 172.21.15.33:0/3855106346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003ab0 con 0x7f79a8106d60 2024-01-25T16:13:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.840+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a81a3f20 con 0x7f79a8106d60 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.842+0000 7f79a4ff9640 1 -- 172.21.15.33:0/3855106346 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f79a0004250 con 0x7f79a8106d60 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 Processor -- start 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 -- start start 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a4ff9640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 0x7f79940a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa6a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.852 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa73240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f9d9070 con 0x558d5fa73780 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f9d6760 con 0x558d5fa6a9b0 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79b0dad740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f9d2050 con 0x558d5fa6a090 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 0x7f79940a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a4ff9640 1 -- 172.21.15.33:0/3855106346 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f79a00ef540 con 0x7f79a8106d60 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa73240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42456/0 (socket says 172.21.15.33:42456) 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a7fff640 1 -- 172.21.15.33:0/437382393 learned_addr learned my addr 172.21.15.33:0/437382393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.843+0000 7f79a77fe640 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa6a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 -- 172.21.15.33:0/437382393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 msgr2=0x558d5fa6a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa6a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 -- 172.21.15.33:0/437382393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 msgr2=0x558d5fa73240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.853 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa73240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 -- 172.21.15.33:0/437382393 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5f9c9e70 con 0x558d5fa73780 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa73240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 0x7f79940a39b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79a80770b0 tx=0x7f79900098b0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f79a7fff640 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b88e0 tx=0x7f79a00bc2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d7894aa3922e90 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f799dffb640 1 -- 172.21.15.33:0/437382393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00c0530 con 0x558d5fa73780 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f799dffb640 1 -- 172.21.15.33:0/437382393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79a00c4770 con 0x558d5fa73780 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.844+0000 7f799dffb640 1 -- 172.21.15.33:0/437382393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00c0170 con 0x558d5fa73780 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- 172.21.15.33:0/437382393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 msgr2=0x558d5fa75b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b88e0 tx=0x7f79a00bc2e0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- 172.21.15.33:0/437382393 shutdown_connections 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa75b70 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa73240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- 172.21.15.33:0/437382393 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa6a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- 172.21.15.33:0/437382393 >> 172.21.15.33:0/437382393 conn(0x558d5fa649b0 msgr2=0x558d5fa64d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.854 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- 172.21.15.33:0/437382393 shutdown_connections 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- 172.21.15.33:0/437382393 wait complete. 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- start start 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 Processor -- start 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa70cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f98b1e0 con 0x558d5fa73780 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f98ae80 con 0x558d5fa6a9b0 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.845+0000 7f79b0dad740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f9500c0 con 0x558d5fa6a090 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79b0dad740 10 client.0 _ll_register_callbacks cb 0x558d5f975ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa70cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42468/0 (socket says 172.21.15.33:42468) 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 -- 172.21.15.33:0/444900707 learned_addr learned my addr 172.21.15.33:0/444900707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:41.855 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a77fe640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 -- 172.21.15.33:0/444900707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 msgr2=0x558d5fa70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 -- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 msgr2=0x558d5fa70cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa70cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 -- 172.21.15.33:0/444900707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a81a3f20 con 0x558d5fa73780 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79a7fff640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009900 tx=0x7f79a002c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d183f9cd1c9498 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f7986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00da030 con 0x558d5fa73780 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f79a00c0530 con 0x558d5fa73780 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d5f9cb040 con 0x558d5fa73780 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a00bc640 con 0x558d5fa73780 2024-01-25T16:13:41.856 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79b0dad740 10 client.15864 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558d5f9d06d0 con 0x558d5fa73780 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.846+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d5f9cb040 con 0x558d5fa73780 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f79a003c070 con 0x558d5fa73780 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 10 client.15864 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 20 client.15864 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fa82ef0 0x558d5fa852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 4 client.15864 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d5f9cad60 con 0x558d5fa82ef0 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79a77fe640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fa82ef0 0x558d5fa852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f79a77fe640 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fa82ef0 0x558d5fa852e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef2b5478cec41d71 server_cookie=a76349ba317036a7 in_seq=0 out_seq=0 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.847+0000 7f7986ffd640 10 client.15864 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.848+0000 7f7986ffd640 1 -- 172.21.15.33:0/444900707 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d5f9cad60 con 0x558d5fa82ef0 2024-01-25T16:13:41.857 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.848+0000 7f7986ffd640 10 client.15864 handle_command_reply: tid=0 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.848+0000 7f79b0dad740 1 client.15864 shutdown 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 msgr2=0x558d5fa711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009900 tx=0x7f79a002c450 comp rx=0 tx=0).stop 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 shutdown_connections 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fa82ef0 0x558d5fa852e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa711f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa70cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 --2- 172.21.15.33:0/444900707 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f7986ffd640 0 client.15864 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 >> 172.21.15.33:0/444900707 conn(0x558d5fa649b0 msgr2=0x558d5fa66700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 shutdown_connections 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 1 -- 172.21.15.33:0/444900707 wait complete. 2024-01-25T16:13:41.858 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.849+0000 7f79b0dad740 20 client.15864 trim_cache size 0 max 16384 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.862+0000 7f79b0dad740 1 Processor -- start 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.862+0000 7f79b0dad740 1 -- start start 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa72ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa73780 0x558d5fa6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.866 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5f9cb070 con 0x558d5fa6a9b0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5fa7fe60 con 0x558d5fa73780 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79b0dad740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5fa60270 con 0x558d5fa6a090 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42478/0 (socket says 172.21.15.33:42478) 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 -- 172.21.15.33:0/3845535713 learned_addr learned my addr 172.21.15.33:0/3845535713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a77fe640 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa72ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa73780 0x558d5fa6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 -- 172.21.15.33:0/3845535713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 msgr2=0x558d5fa72ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa72ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 -- 172.21.15.33:0/3845535713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa73780 msgr2=0x558d5fa6d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa73780 0x558d5fa6d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 -- 172.21.15.33:0/3845535713 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d5f96d060 con 0x558d5fa6a9b0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7990009d60 tx=0x7f7990005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f421189181275b1c server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.867 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f799d7fa640 1 -- 172.21.15.33:0/3845535713 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79900130b0 con 0x558d5fa6a9b0 2024-01-25T16:13:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f799d7fa640 1 -- 172.21.15.33:0/3845535713 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7990011e80 con 0x558d5fa6a9b0 2024-01-25T16:13:41.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.863+0000 7f799d7fa640 1 -- 172.21.15.33:0/3845535713 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799002ba30 con 0x558d5fa6a9b0 2024-01-25T16:13:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- 172.21.15.33:0/3845535713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 msgr2=0x558d5fa71010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7990009d60 tx=0x7f7990005f90 comp rx=0 tx=0).stop 2024-01-25T16:13:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- 172.21.15.33:0/3845535713 shutdown_connections 2024-01-25T16:13:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa73780 0x558d5fa6d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa6a9b0 0x558d5fa71010 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3845535713 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa72ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- 172.21.15.33:0/3845535713 >> 172.21.15.33:0/3845535713 conn(0x558d5fa649b0 msgr2=0x558d5fa6f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- 172.21.15.33:0/3845535713 shutdown_connections 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- 172.21.15.33:0/3845535713 wait complete. 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.864+0000 7f79b0dad740 1 -- start start 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 Processor -- start 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa78ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa79220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5fad9370 con 0x558d5fa73780 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5fad94b0 con 0x558d5fa6a9b0 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d5fad95f0 con 0x558d5fa6a090 2024-01-25T16:13:41.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79b0dad740 10 client.0 _ll_register_callbacks cb 0x7f79a00b8060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42480/0 (socket says 172.21.15.33:42480) 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 -- 172.21.15.33:0/3125610433 learned_addr learned my addr 172.21.15.33:0/3125610433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a77fe640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa78ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a6ffd640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa79220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 -- 172.21.15.33:0/3125610433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 msgr2=0x558d5fa78ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa78ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 -- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 msgr2=0x558d5fa79220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa79220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 -- 172.21.15.33:0/3125610433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7990009e70 con 0x558d5fa73780 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f79a7fff640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b8e50 tx=0x7f79a00ba990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b5b42addb821713 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.865+0000 7f7986ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00da040 con 0x558d5fa73780 2024-01-25T16:13:41.873 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f79a00c0530 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d5f6d1600 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a00bcae0 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x558d5f992a80 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d5f6d1600 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f79a003c070 con 0x558d5fa73780 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 20 client.15870 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fada8c0 0x558d5fadccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 4 client.15870 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558d5f724c50 con 0x558d5fada8c0 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.866+0000 7f79a77fe640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fada8c0 0x558d5fadccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:41.874 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.867+0000 7f7986ffd640 10 client.15870 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:41.875 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.867+0000 7f79a77fe640 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fada8c0 0x558d5fadccb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226557f24d4ce888 server_cookie=ce01d3537bae3470 in_seq=0 out_seq=0 2024-01-25T16:13:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.964+0000 7f7986ffd640 1 -- 172.21.15.33:0/3125610433 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558d5f724c50 con 0x558d5fada8c0 2024-01-25T16:13:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.964+0000 7f7986ffd640 10 client.15870 handle_command_reply: tid=0 2024-01-25T16:13:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.964+0000 7f79b0dad740 1 client.15870 shutdown 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.964+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 msgr2=0x558d5fa79760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.964+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b8e50 tx=0x7f79a00ba990 comp rx=0 tx=0).stop 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 shutdown_connections 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x558d5fada8c0 0x558d5fadccb0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f7986ffd640 0 client.15870 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558d5fa73780 0x558d5fa79760 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558d5fa6a9b0 0x558d5fa79220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 --2- 172.21.15.33:0/3125610433 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x558d5fa6a090 0x558d5fa78ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 >> 172.21.15.33:0/3125610433 conn(0x558d5fa649b0 msgr2=0x558d5fa63a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 shutdown_connections 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 1 -- 172.21.15.33:0/3125610433 wait complete. 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.965+0000 7f79b0dad740 20 client.15870 trim_cache size 0 max 16384 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:13:41.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 msgr2=0x7f79940a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 0x7f79940a39b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f79a80770b0 tx=0x7f79900098b0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 msgr2=0x7f79a819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f79a0005940 tx=0x7f79a002ccd0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 shutdown_connections 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f79940a14f0 0x7f79940a39b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79a8106d60 0x7f79a819ba90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.970 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f79a8068cf0 0x7f79a8197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 --2- 172.21.15.33:0/3855106346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79a8068370 0x7f79a81971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 >> 172.21.15.33:0/3855106346 conn(0x7f79a8076a40 msgr2=0x7f79a80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 shutdown_connections 2024-01-25T16:13:41.971 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:41.966+0000 7f79ae1e5640 1 -- 172.21.15.33:0/3855106346 wait complete. 2024-01-25T16:13:42.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 179755 4.09 MB/sec execute 159 sec latency 40.875 ms 2024-01-25T16:13:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:43 smithi033 ceph-mon[31417]: pgmap v180: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-01-25T16:13:43.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:43 smithi152 ceph-mon[36015]: pgmap v180: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-01-25T16:13:43.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:43 smithi076 ceph-mon[35642]: pgmap v180: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 370 op/s 2024-01-25T16:13:43.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 181796 4.14 MB/sec execute 160 sec latency 13.212 ms 2024-01-25T16:13:44.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 184108 4.20 MB/sec execute 161 sec latency 53.891 ms 2024-01-25T16:13:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:45 smithi033 ceph-mon[31417]: pgmap v181: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-01-25T16:13:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:45 smithi152 ceph-mon[36015]: pgmap v181: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-01-25T16:13:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:45 smithi076 ceph-mon[35642]: pgmap v181: 97 pgs: 97 active+clean; 71 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 193 op/s 2024-01-25T16:13:45.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 185975 4.24 MB/sec execute 162 sec latency 43.343 ms 2024-01-25T16:13:46.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 188345 4.30 MB/sec execute 163 sec latency 24.812 ms 2024-01-25T16:13:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:47 smithi033 ceph-mon[31417]: pgmap v182: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:47.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:47 smithi152 ceph-mon[36015]: pgmap v182: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:47 smithi076 ceph-mon[35642]: pgmap v182: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.3 MiB/s wr, 359 op/s 2024-01-25T16:13:47.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 190449 4.32 MB/sec execute 164 sec latency 18.203 ms 2024-01-25T16:13:48.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:48 smithi033 ceph-mon[31417]: pgmap v183: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:48.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:48 smithi152 ceph-mon[36015]: pgmap v183: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:48.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:48 smithi076 ceph-mon[35642]: pgmap v183: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:48.774 INFO:tasks.workunit.client.0.smithi033.stdout: 1 192525 4.37 MB/sec execute 165 sec latency 16.805 ms 2024-01-25T16:13:49.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 195213 4.44 MB/sec execute 166 sec latency 13.841 ms 2024-01-25T16:13:50.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 197237 4.46 MB/sec execute 167 sec latency 16.637 ms 2024-01-25T16:13:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:51 smithi033 ceph-mon[31417]: pgmap v184: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:51.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:51 smithi152 ceph-mon[36015]: pgmap v184: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:51 smithi076 ceph-mon[35642]: pgmap v184: 97 pgs: 97 active+clean; 59 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.7 MiB/s wr, 264 op/s 2024-01-25T16:13:51.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 199173 4.51 MB/sec execute 168 sec latency 40.815 ms 2024-01-25T16:13:52.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 201532 4.56 MB/sec execute 169 sec latency 30.574 ms 2024-01-25T16:13:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:53 smithi033 ceph-mon[31417]: pgmap v185: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-01-25T16:13:53.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:53 smithi152 ceph-mon[36015]: pgmap v185: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-01-25T16:13:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:53 smithi076 ceph-mon[35642]: pgmap v185: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.5 MiB/s wr, 354 op/s 2024-01-25T16:13:53.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 203571 4.60 MB/sec execute 170 sec latency 18.382 ms 2024-01-25T16:13:54.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 205978 4.65 MB/sec execute 171 sec latency 15.769 ms 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 -- 172.21.15.33:0/3508775821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00ffe40 msgr2=0x7fd4f01002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 --2- 172.21.15.33:0/3508775821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00ffe40 0x7fd4f01002c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009900 tx=0x7fd4e002bc30 comp rx=0 tx=0).stop 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 -- 172.21.15.33:0/3508775821 shutdown_connections 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 --2- 172.21.15.33:0/3508775821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f0100800 0x7fd4f010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 --2- 172.21.15.33:0/3508775821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00ffe40 0x7fd4f01002c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.058 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 --2- 172.21.15.33:0/3508775821 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0104b90 0x7fd4f0104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 -- 172.21.15.33:0/3508775821 >> 172.21.15.33:0/3508775821 conn(0x7fd4f0076a40 msgr2=0x7fd4f0076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 -- 172.21.15.33:0/3508775821 shutdown_connections 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.052+0000 7fd4f7789640 1 -- 172.21.15.33:0/3508775821 wait complete. 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 Processor -- start 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 -- start start 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f00ffe40 0x7fd4f0197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0100800 0x7fd4f01977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f019c120 con 0x7fd4f0104b90 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f019c290 con 0x7fd4f00ffe40 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f7789640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f019c450 con 0x7fd4f0100800 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.059 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f4d24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0100800 0x7fd4f01977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58712/0 (socket says 172.21.15.33:58712) 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f5d26640 1 -- 172.21.15.33:0/1437919338 learned_addr learned my addr 172.21.15.33:0/1437919338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.053+0000 7fd4f5525640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f00ffe40 0x7fd4f0197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 -- 172.21.15.33:0/1437919338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0100800 msgr2=0x7fd4f01977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0100800 0x7fd4f01977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 -- 172.21.15.33:0/1437919338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f00ffe40 msgr2=0x7fd4f0197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f00ffe40 0x7fd4f0197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 -- 172.21.15.33:0/1437919338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009560 con 0x7fd4f0104b90 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800ea60 tx=0x7fd4e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4e67fc640 1 -- 172.21.15.33:0/1437919338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800ce40 con 0x7fd4f0104b90 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4e67fc640 1 -- 172.21.15.33:0/1437919338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4e8005020 con 0x7fd4f0104b90 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4e67fc640 1 -- 172.21.15.33:0/1437919338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8010670 con 0x7fd4f0104b90 2024-01-25T16:13:55.060 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f019c730 con 0x7fd4f0104b90 2024-01-25T16:13:55.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.054+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f01a3fb0 con 0x7fd4f0104b90 2024-01-25T16:13:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.056+0000 7fd4e67fc640 1 -- 172.21.15.33:0/1437919338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd4e8005450 con 0x7fd4f0104b90 2024-01-25T16:13:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 Processor -- start 2024-01-25T16:13:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 -- start start 2024-01-25T16:13:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4e67fc640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 0x7fd4dc0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x56181580a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x56181580cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815770070 con 0x56181580a780 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56181576d760 con 0x5618158019b0 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4fa424740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815769050 con 0x561815801090 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 0x7fd4dc0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4e67fc640 1 -- 172.21.15.33:0/1437919338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd4e8014070 con 0x7fd4f0104b90 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:44462/0 (socket says 172.21.15.33:44462) 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x56181580cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 -- 172.21.15.33:0/2685645016 learned_addr learned my addr 172.21.15.33:0/2685645016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x56181580a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 -- 172.21.15.33:0/2685645016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 msgr2=0x56181580a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x56181580a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 -- 172.21.15.33:0/2685645016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 msgr2=0x56181580cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x56181580cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5525640 1 -- 172.21.15.33:0/2685645016 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561815760e70 con 0x561815801090 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x56181580cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.057+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 0x7fd4dc0a39b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006010 tx=0x7fd4e00049b0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00bd10 tx=0x7fd4ec0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2b4ef453ec185af server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4db7fe640 1 -- 172.21.15.33:0/2685645016 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec004030 con 0x561815801090 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4db7fe640 1 -- 172.21.15.33:0/2685645016 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4ec015070 con 0x561815801090 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4db7fe640 1 -- 172.21.15.33:0/2685645016 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec0116a0 con 0x561815801090 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 -- 172.21.15.33:0/2685645016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 msgr2=0x561815801470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00bd10 tx=0x7fd4ec0107a0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 -- 172.21.15.33:0/2685645016 shutdown_connections 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x56181580cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x56181580a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2685645016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815801470 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 -- 172.21.15.33:0/2685645016 >> 172.21.15.33:0/2685645016 conn(0x5618157fb9b0 msgr2=0x5618157fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 -- 172.21.15.33:0/2685645016 shutdown_connections 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.058+0000 7fd4fa424740 1 -- 172.21.15.33:0/2685645016 wait complete. 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 -- start start 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 Processor -- start 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815807c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618157221e0 con 0x56181580a780 2024-01-25T16:13:55.069 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815721e80 con 0x561815801090 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618156e70c0 con 0x5618158019b0 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4fa424740 10 client.0 _ll_register_callbacks cb 0x56181570cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58724/0 (socket says 172.21.15.33:58724) 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f4d24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815807c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f5d26640 1 -- 172.21.15.33:0/2393822239 learned_addr learned my addr 172.21.15.33:0/2393822239 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.059+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:53736/0 (socket says 172.21.15.33:53736) 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 -- 172.21.15.33:0/2393822239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 msgr2=0x561815807c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815807c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 -- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 msgr2=0x561815807740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.070 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 -- 172.21.15.33:0/2393822239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec00b940 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815807c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cabb0 tx=0x7fd4e809a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1150b4454ec29a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80a2590 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd4e80ade40 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561815762040 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4e80b8250 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4fa424740 10 client.15882 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5618157676d0 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561815762040 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.060+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd4e801f060 con 0x56181580a780 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 10 client.15882 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 20 client.15882 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56181581a680 0x56181581caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 4 client.15882 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561815761d60 con 0x56181581a680 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56181581a680 0x56181581caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4f5525640 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56181581a680 0x56181581caa0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7607643fe6717bd0 server_cookie=f978174c90a6b3a4 in_seq=0 out_seq=0 2024-01-25T16:13:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.061+0000 7fd4d8ff9640 10 client.15882 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.063+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/2393822239 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561815761d60 con 0x56181581a680 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.063+0000 7fd4d8ff9640 10 client.15882 handle_command_reply: tid=0 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.063+0000 7fd4fa424740 1 client.15882 shutdown 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.063+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 msgr2=0x5618158081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.063+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80cabb0 tx=0x7fd4e809a2c0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 shutdown_connections 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56181581a680 0x56181581caa0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158081c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815807c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4d8ff9640 0 client.15882 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 --2- 172.21.15.33:0/2393822239 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561815801090 0x561815807740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 >> 172.21.15.33:0/2393822239 conn(0x5618157fb9b0 msgr2=0x5618157fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 shutdown_connections 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 1 -- 172.21.15.33:0/2393822239 wait complete. 2024-01-25T16:13:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.064+0000 7fd4fa424740 20 client.15882 trim_cache size 0 max 16384 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 Processor -- start 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 -- start start 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815807fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x561815811470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815762040 con 0x56181580a780 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618157f72e0 con 0x5618158019b0 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4fa424740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618157fc9b0 con 0x561815801090 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.077+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58726/0 (socket says 172.21.15.33:58726) 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 -- 172.21.15.33:0/3529601770 learned_addr learned my addr 172.21.15.33:0/3529601770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5525640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815807fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x561815811470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 -- 172.21.15.33:0/3529601770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 msgr2=0x561815807fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815807fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 -- 172.21.15.33:0/3529601770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 msgr2=0x561815811470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x561815811470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 -- 172.21.15.33:0/3529601770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561815704060 con 0x56181580a780 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5525640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815807fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:13:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4e809a190 tx=0x7fd4e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bdcb3985defc67 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4daffd640 1 -- 172.21.15.33:0/3529601770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e80996f0 con 0x56181580a780 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4daffd640 1 -- 172.21.15.33:0/3529601770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4e8099d40 con 0x56181580a780 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.078+0000 7fd4daffd640 1 -- 172.21.15.33:0/3529601770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8096910 con 0x56181580a780 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- 172.21.15.33:0/3529601770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 msgr2=0x5618158119b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd4e809a190 tx=0x7fd4e8005850 comp rx=0 tx=0).stop 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- 172.21.15.33:0/3529601770 shutdown_connections 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56181580a780 0x5618158119b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5618158019b0 0x561815811470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- 172.21.15.33:0/3529601770 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561815801090 0x561815807fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- 172.21.15.33:0/3529601770 >> 172.21.15.33:0/3529601770 conn(0x5618157fb9b0 msgr2=0x5618157fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- 172.21.15.33:0/3529601770 shutdown_connections 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- 172.21.15.33:0/3529601770 wait complete. 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- start start 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 Processor -- start 2024-01-25T16:13:55.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815810570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 0x561815810ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815870bb0 con 0x561815801090 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815870cf0 con 0x56181580a780 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.079+0000 7fd4fa424740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561815870e60 con 0x5618158019b0 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4fa424740 10 client.0 _ll_register_callbacks cb 0x7fd4ec00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58738/0 (socket says 172.21.15.33:58738) 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 -- 172.21.15.33:0/941955132 learned_addr learned my addr 172.21.15.33:0/941955132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 0x561815810ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815810570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 -- 172.21.15.33:0/941955132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 msgr2=0x561815810570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815810570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 -- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 msgr2=0x561815810ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 0x561815810ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 -- 172.21.15.33:0/941955132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618157676d0 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5d26640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 0x561815810ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4f5525640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec011010 tx=0x7fd4ec00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8de4f5092f53c6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec020040 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd4ec017070 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4ec015070 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4ec003a00 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4fa424740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.080+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561815729a80 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4ec015070 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd4ec028430 con 0x561815801090 2024-01-25T16:13:55.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 20 client.15888 populate_metadata read hostname 'smithi033' 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5618158725d0 0x5618158749c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 4 client.15888 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561815468600 con 0x5618158725d0 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5618158725d0 0x5618158749c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4f4d24640 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5618158725d0 0x5618158749c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2315e8a65a62d945 server_cookie=4e94e88fea0e40d4 in_seq=0 out_seq=0 2024-01-25T16:13:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.081+0000 7fd4d8ff9640 10 client.15888 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4d8ff9640 1 -- 172.21.15.33:0/941955132 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561815468600 con 0x5618158725d0 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4d8ff9640 10 client.15888 handle_command_reply: tid=0 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 client.15888 shutdown 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 msgr2=0x561815810030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec011010 tx=0x7fd4ec00b820 comp rx=0 tx=0).stop 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 shutdown_connections 2024-01-25T16:13:55.220 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5618158725d0 0x5618158749c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56181580a780 0x561815810ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5618158019b0 0x561815810570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 --2- 172.21.15.33:0/941955132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561815801090 0x561815810030 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 >> 172.21.15.33:0/941955132 conn(0x5618157fb9b0 msgr2=0x5618157faa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4d8ff9640 0 client.15888 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 shutdown_connections 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 1 -- 172.21.15.33:0/941955132 wait complete. 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.216+0000 7fd4fa424740 20 client.15888 trim_cache size 0 max 16384 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:13:55.221 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:13:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 msgr2=0x7fd4dc0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 0x7fd4dc0a39b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0006010 tx=0x7fd4e00049b0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.222 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 msgr2=0x7fd4f019bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800ea60 tx=0x7fd4e800ef20 comp rx=0 tx=0).stop 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 shutdown_connections 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd4dc0a14f0 0x7fd4dc0a39b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f0104b90 0x7fd4f019bb20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd4f0100800 0x7fd4f01977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 --2- 172.21.15.33:0/1437919338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4f00ffe40 0x7fd4f0197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 >> 172.21.15.33:0/1437919338 conn(0x7fd4f0076a40 msgr2=0x7fd4f00685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 shutdown_connections 2024-01-25T16:13:55.223 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:13:55.217+0000 7fd4f7789640 1 -- 172.21.15.33:0/1437919338 wait complete. 2024-01-25T16:13:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:55 smithi033 ceph-mon[31417]: pgmap v186: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-01-25T16:13:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:55 smithi152 ceph-mon[36015]: pgmap v186: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-01-25T16:13:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:55 smithi076 ceph-mon[35642]: pgmap v186: 97 pgs: 97 active+clean; 66 MiB data, 874 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-01-25T16:13:55.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 207555 4.66 MB/sec execute 172 sec latency 27.333 ms 2024-01-25T16:13:56.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 209491 4.71 MB/sec execute 173 sec latency 158.248 ms 2024-01-25T16:13:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:57 smithi033 ceph-mon[31417]: pgmap v187: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 462 op/s 2024-01-25T16:13:57.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:57 smithi152 ceph-mon[36015]: pgmap v187: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 462 op/s 2024-01-25T16:13:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:57 smithi076 ceph-mon[35642]: pgmap v187: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.0 MiB/s wr, 462 op/s 2024-01-25T16:13:57.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 211258 4.72 MB/sec execute 174 sec latency 74.979 ms 2024-01-25T16:13:58.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213301 4.77 MB/sec execute 175 sec latency 53.383 ms 2024-01-25T16:13:59.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:13:59 smithi076 ceph-mon[35642]: pgmap v188: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:13:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:13:59 smithi033 ceph-mon[31417]: pgmap v188: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:13:59.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:13:59 smithi152 ceph-mon[36015]: pgmap v188: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:13:59.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 215342 4.79 MB/sec execute 176 sec latency 85.262 ms 2024-01-25T16:14:00.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:00 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:14:00.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:00 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:14:00.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:00 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:14:00.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 216811 4.82 MB/sec execute 177 sec latency 26.805 ms 2024-01-25T16:14:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:01 smithi033 ceph-mon[31417]: pgmap v189: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:14:01.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:01 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:14:01.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:01 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:14:01.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:01 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:14:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:01 smithi152 ceph-mon[36015]: pgmap v189: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:14:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:01 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:14:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:01 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:14:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:01 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:14:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:01 smithi076 ceph-mon[35642]: pgmap v189: 97 pgs: 97 active+clean; 56 MiB data, 894 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 296 op/s 2024-01-25T16:14:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:01 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:14:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:01 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:14:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:01 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:14:01.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 218563 4.83 MB/sec execute 178 sec latency 176.276 ms 2024-01-25T16:14:02.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 220382 4.88 MB/sec execute 179 sec latency 186.116 ms 2024-01-25T16:14:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:03 smithi033 ceph-mon[31417]: pgmap v190: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-01-25T16:14:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:03 smithi033 ceph-mon[31417]: osdmap e91: 12 total, 12 up, 12 in 2024-01-25T16:14:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:03 smithi152 ceph-mon[36015]: pgmap v190: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-01-25T16:14:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:03 smithi152 ceph-mon[36015]: osdmap e91: 12 total, 12 up, 12 in 2024-01-25T16:14:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:03 smithi076 ceph-mon[35642]: pgmap v190: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.9 MiB/s wr, 434 op/s 2024-01-25T16:14:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:03 smithi076 ceph-mon[35642]: osdmap e91: 12 total, 12 up, 12 in 2024-01-25T16:14:03.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 222572 4.89 MB/sec execute 180 sec latency 19.245 ms 2024-01-25T16:14:04.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 224772 4.94 MB/sec execute 181 sec latency 17.393 ms 2024-01-25T16:14:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:05 smithi033 ceph-mon[31417]: pgmap v192: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-01-25T16:14:05.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:05 smithi152 ceph-mon[36015]: pgmap v192: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-01-25T16:14:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:05 smithi076 ceph-mon[35642]: pgmap v192: 97 pgs: 97 active+clean; 62 MiB data, 922 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 414 op/s 2024-01-25T16:14:05.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 227304 5.00 MB/sec execute 182 sec latency 15.787 ms 2024-01-25T16:14:06.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 229352 5.02 MB/sec execute 183 sec latency 37.932 ms 2024-01-25T16:14:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:07 smithi033 ceph-mon[31417]: pgmap v193: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:07 smithi152 ceph-mon[36015]: pgmap v193: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:07 smithi076 ceph-mon[35642]: pgmap v193: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:07.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 231162 5.06 MB/sec execute 184 sec latency 76.242 ms 2024-01-25T16:14:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- 172.21.15.33:0/4208007277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14106000 msgr2=0x7fec1410dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- 172.21.15.33:0/4208007277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14106000 0x7fec1410dbd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009cd0 tx=0x7fec0c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- 172.21.15.33:0/4208007277 shutdown_connections 2024-01-25T16:14:08.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- 172.21.15.33:0/4208007277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14106000 0x7fec1410dbd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- 172.21.15.33:0/4208007277 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec141018e0 0x7fec14105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- 172.21.15.33:0/4208007277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14100f30 0x7fec14101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- 172.21.15.33:0/4208007277 >> 172.21.15.33:0/4208007277 conn(0x7fec140fc7d0 msgr2=0x7fec140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- 172.21.15.33:0/4208007277 shutdown_connections 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- 172.21.15.33:0/4208007277 wait complete. 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 Processor -- start 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- start start 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14106000 0x7fec14197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1419ba50 0x7fec1419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c440 con 0x7fec14100f30 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c5b0 con 0x7fec14106000 2024-01-25T16:14:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.310+0000 7fec1a901640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1419c720 con 0x7fec1419ba50 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34480/0 (socket says 172.21.15.33:34480) 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 -- 172.21.15.33:0/3683915491 learned_addr learned my addr 172.21.15.33:0/3683915491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec18e9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1419ba50 0x7fec1419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14106000 0x7fec14197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 -- 172.21.15.33:0/3683915491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1419ba50 msgr2=0x7fec1419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1419ba50 0x7fec1419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 -- 172.21.15.33:0/3683915491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14106000 msgr2=0x7fec14197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14106000 0x7fec14197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 -- 172.21.15.33:0/3683915491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec0c009900 con 0x7fec14100f30 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec13fff640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec11ffb640 1 -- 172.21.15.33:0/3683915491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04009700 con 0x7fec14100f30 2024-01-25T16:14:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec11ffb640 1 -- 172.21.15.33:0/3683915491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec0400cd80 con 0x7fec14100f30 2024-01-25T16:14:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec141a3d00 con 0x7fec14100f30 2024-01-25T16:14:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec11ffb640 1 -- 172.21.15.33:0/3683915491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec04018860 con 0x7fec14100f30 2024-01-25T16:14:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.311+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141a41b0 con 0x7fec14100f30 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.313+0000 7fec11ffb640 1 -- 172.21.15.33:0/3683915491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fec040189c0 con 0x7fec14100f30 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 Processor -- start 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 -- start start 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 0x5556486f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555648653070 con 0x5556486e4450 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555648650760 con 0x5556486f1b90 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec1d59b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564864c050 con 0x5556486e3b30 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec18e9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 0x5556486f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec11ffb640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 0x7fec000a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec18e9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:36716/0 (socket says 172.21.15.33:36716) 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec18e9e640 1 -- 172.21.15.33:0/482764928 learned_addr learned my addr 172.21.15.33:0/482764928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34496/0 (socket says 172.21.15.33:34496) 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec18e9e640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 0x7fec000a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 -- 172.21.15.33:0/482764928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 msgr2=0x5556486e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 -- 172.21.15.33:0/482764928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 msgr2=0x5556486f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec11ffb640 1 -- 172.21.15.33:0/3683915491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fec04014070 con 0x7fec14100f30 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec0bfff640 1 -- 172.21.15.33:0/482764928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555648643e70 con 0x5556486f1b90 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.314+0000 7fec18e9e640 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec0bfff640 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 0x5556486f3f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec1406a320 tx=0x7febfc00ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e3e67a9cd197a1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec18e9e640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 0x7fec000a3a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec0c033000 tx=0x7fec0c02c0f0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec0a7fc640 1 -- 172.21.15.33:0/482764928 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc003a90 con 0x5556486f1b90 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec0a7fc640 1 -- 172.21.15.33:0/482764928 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febfc0054e0 con 0x5556486f1b90 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec0a7fc640 1 -- 172.21.15.33:0/482764928 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc004d40 con 0x5556486f1b90 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec1d59b740 1 -- 172.21.15.33:0/482764928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 msgr2=0x5556486f3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.315+0000 7fec1d59b740 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 0x5556486f3f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec1406a320 tx=0x7febfc00ba60 comp rx=0 tx=0).stop 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- 172.21.15.33:0/482764928 shutdown_connections 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486f1b90 0x5556486f3f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- 172.21.15.33:0/482764928 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x5556486e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- 172.21.15.33:0/482764928 >> 172.21.15.33:0/482764928 conn(0x5556486de9b0 msgr2=0x5556486ded90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- 172.21.15.33:0/482764928 shutdown_connections 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- 172.21.15.33:0/482764928 wait complete. 2024-01-25T16:14:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- start start 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 Processor -- start 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486eea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486051e0 con 0x5556486e4450 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555648604e80 con 0x5556486e3b30 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556485ca0c0 con 0x5556486f1b90 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.316+0000 7fec1d59b740 10 client.0 _ll_register_callbacks cb 0x5556485efac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34508/0 (socket says 172.21.15.33:34508) 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 -- 172.21.15.33:0/3027627240 learned_addr learned my addr 172.21.15.33:0/3027627240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec13fff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec18e9e640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 -- 172.21.15.33:0/3027627240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 msgr2=0x5556486ef4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486ef4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 -- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 msgr2=0x5556486eea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486eea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 -- 172.21.15.33:0/3027627240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febfc004210 con 0x5556486e4450 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec13fff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bed7423951469e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7febf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc01f070 con 0x5556486e4450 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7febfc003e40 con 0x5556486e4450 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555648645040 con 0x5556486e4450 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7febfc004a80 con 0x5556486e4450 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec1d59b740 10 client.15900 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55564864a6d0 con 0x5556486e4450 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.317+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555648645040 con 0x5556486e4450 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febfc011880 con 0x5556486e4450 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 10 client.15900 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 20 client.15900 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648701890 0x555648703cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 4 client.15900 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555648644d60 con 0x555648701890 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec13fff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648701890 0x555648703cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7fec13fff640 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648701890 0x555648703cb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1491d935944f80b7 server_cookie=2439ad307c4c89fa in_seq=0 out_seq=0 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.318+0000 7febf37fe640 10 client.15900 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7febf37fe640 1 -- 172.21.15.33:0/3027627240 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555648644d60 con 0x555648701890 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7febf37fe640 10 client.15900 handle_command_reply: tid=0 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 client.15900 shutdown 2024-01-25T16:14:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 msgr2=0x5556486eef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc003ac0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 shutdown_connections 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648701890 0x555648703cb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486ef4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7febf37fe640 0 client.15900 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486eef90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3027627240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486eea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.319+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 >> 172.21.15.33:0/3027627240 conn(0x5556486de9b0 msgr2=0x5556486e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.320+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 shutdown_connections 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.320+0000 7fec1d59b740 1 -- 172.21.15.33:0/3027627240 wait complete. 2024-01-25T16:14:08.330 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.320+0000 7fec1d59b740 20 client.15900 trim_cache size 0 max 16384 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 Processor -- start 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 -- start start 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486da0e0 con 0x5556486e4450 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486ea740 con 0x5556486e3b30 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec1d59b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486de860 con 0x5556486f1b90 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34518/0 (socket says 172.21.15.33:34518) 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 -- 172.21.15.33:0/3506899344 learned_addr learned my addr 172.21.15.33:0/3506899344 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec13fff640 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec18e9e640 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 -- 172.21.15.33:0/3506899344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 msgr2=0x5556486f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486f8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 -- 172.21.15.33:0/3506899344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 msgr2=0x5556486ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.333+0000 7fec0bfff640 1 -- 172.21.15.33:0/3506899344 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556485e7060 con 0x5556486e4450 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7febfc004810 tx=0x7febfc005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c91d55e2273bdf server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec09ffb640 1 -- 172.21.15.33:0/3506899344 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc01f040 con 0x5556486e4450 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec09ffb640 1 -- 172.21.15.33:0/3506899344 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febfc005710 con 0x5556486e4450 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec09ffb640 1 -- 172.21.15.33:0/3506899344 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc019b30 con 0x5556486e4450 2024-01-25T16:14:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 -- 172.21.15.33:0/3506899344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 msgr2=0x5556486f88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7febfc004810 tx=0x7febfc005c30 comp rx=0 tx=0).stop 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 -- 172.21.15.33:0/3506899344 shutdown_connections 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486f1b90 0x5556486f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486e4450 0x5556486f88c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3506899344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e3b30 0x5556486ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 -- 172.21.15.33:0/3506899344 >> 172.21.15.33:0/3506899344 conn(0x5556486de9b0 msgr2=0x5556486dd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 -- 172.21.15.33:0/3506899344 shutdown_connections 2024-01-25T16:14:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.334+0000 7fec1d59b740 1 -- 172.21.15.33:0/3506899344 wait complete. 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 -- start start 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 Processor -- start 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e4450 0x555648752650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486f8110 con 0x5556486f1b90 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486f8280 con 0x5556486e4450 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556486f8440 con 0x5556486e3b30 2024-01-25T16:14:08.345 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec1d59b740 10 client.0 _ll_register_callbacks cb 0x7febfc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec18e9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec13fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec13fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:36748/0 (socket says 172.21.15.33:36748) 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec18e9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34522/0 (socket says 172.21.15.33:34522) 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec13fff640 1 -- 172.21.15.33:0/3567843936 learned_addr learned my addr 172.21.15.33:0/3567843936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.335+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e4450 0x555648752650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 -- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e4450 msgr2=0x555648752650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e4450 0x555648752650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 -- 172.21.15.33:0/3567843936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 msgr2=0x5556486f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 -- 172.21.15.33:0/3567843936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55564864a6d0 con 0x5556486e3b30 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec18e9e640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec13fff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec0401e000 tx=0x7fec0401f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947d58c710aeb49 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040963d0 con 0x5556486e3b30 2024-01-25T16:14:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febfc0054e0 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fec0409a090 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec0409e130 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 10 client.25462 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55564860ca80 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec040a8080 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec04099610 con 0x5556486e3b30 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 10 client.25462 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 10 client.25462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 20 client.25462 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648759fd0 0x55564875c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 4 client.25462 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.336+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55564834b600 con 0x555648759fd0 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.337+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648759fd0 0x55564875c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.337+0000 7febf37fe640 10 client.25462 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.337+0000 7fec0bfff640 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648759fd0 0x55564875c3c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef897bdd1823acb9 server_cookie=6f02ce380657fe4d in_seq=0 out_seq=0 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7febf37fe640 1 -- 172.21.15.33:0/3567843936 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55564834b600 con 0x555648759fd0 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7febf37fe640 10 client.25462 handle_command_reply: tid=0 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 client.25462 shutdown 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 msgr2=0x555648752110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec0401e000 tx=0x7fec0401f000 comp rx=0 tx=0).stop 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 shutdown_connections 2024-01-25T16:14:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555648759fd0 0x55564875c3c0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5556486f1b90 0x5556486f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5556486e4450 0x555648752650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7febf37fe640 0 client.25462 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 --2- 172.21.15.33:0/3567843936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5556486e3b30 0x555648752110 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 >> 172.21.15.33:0/3567843936 conn(0x5556486de9b0 msgr2=0x5556486dda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 shutdown_connections 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 1 -- 172.21.15.33:0/3567843936 wait complete. 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.449+0000 7fec1d59b740 20 client.25462 trim_cache size 0 max 16384 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:14:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:14:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:14:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:14:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 msgr2=0x7fec000a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 0x7fec000a3a30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec0c033000 tx=0x7fec0c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 msgr2=0x7fec141971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fec0400e9f0 tx=0x7fec0400eeb0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 shutdown_connections 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fec000a1570 0x7fec000a3a30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fec1419ba50 0x7fec1419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec14106000 0x7fec14197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 --2- 172.21.15.33:0/3683915491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec14100f30 0x7fec141971e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 >> 172.21.15.33:0/3683915491 conn(0x7fec140fc7d0 msgr2=0x7fec140feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 shutdown_connections 2024-01-25T16:14:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:08.450+0000 7fec1a901640 1 -- 172.21.15.33:0/3683915491 wait complete. 2024-01-25T16:14:08.775 INFO:tasks.workunit.client.0.smithi033.stdout: 1 233166 5.07 MB/sec execute 185 sec latency 95.147 ms 2024-01-25T16:14:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:09 smithi033 ceph-mon[31417]: pgmap v194: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:09.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:09 smithi152 ceph-mon[36015]: pgmap v194: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:09 smithi076 ceph-mon[35642]: pgmap v194: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:09.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 235206 5.11 MB/sec execute 186 sec latency 19.236 ms 2024-01-25T16:14:10.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 237838 5.17 MB/sec execute 187 sec latency 16.240 ms 2024-01-25T16:14:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:11 smithi033 ceph-mon[31417]: pgmap v195: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:11 smithi033 ceph-mon[31417]: osdmap e92: 12 total, 12 up, 12 in 2024-01-25T16:14:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:11 smithi152 ceph-mon[36015]: pgmap v195: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:11 smithi152 ceph-mon[36015]: osdmap e92: 12 total, 12 up, 12 in 2024-01-25T16:14:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:11 smithi076 ceph-mon[35642]: pgmap v195: 97 pgs: 97 active+clean; 68 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 291 op/s 2024-01-25T16:14:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:11 smithi076 ceph-mon[35642]: osdmap e92: 12 total, 12 up, 12 in 2024-01-25T16:14:11.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 239675 5.18 MB/sec execute 188 sec latency 26.971 ms 2024-01-25T16:14:12.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 241647 5.22 MB/sec execute 189 sec latency 59.192 ms 2024-01-25T16:14:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:13 smithi033 ceph-mon[31417]: pgmap v197: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-01-25T16:14:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:13 smithi152 ceph-mon[36015]: pgmap v197: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-01-25T16:14:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:13 smithi076 ceph-mon[35642]: pgmap v197: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 352 op/s 2024-01-25T16:14:13.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 243793 5.24 MB/sec execute 190 sec latency 18.383 ms 2024-01-25T16:14:14.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 245846 5.28 MB/sec execute 191 sec latency 15.411 ms 2024-01-25T16:14:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:15 smithi033 ceph-mon[31417]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 351 op/s 2024-01-25T16:14:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:15 smithi152 ceph-mon[36015]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 351 op/s 2024-01-25T16:14:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:15 smithi076 ceph-mon[35642]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.7 MiB/s wr, 351 op/s 2024-01-25T16:14:15.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 248440 5.34 MB/sec execute 192 sec latency 16.134 ms 2024-01-25T16:14:16.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 250155 5.34 MB/sec execute 193 sec latency 23.211 ms 2024-01-25T16:14:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:17 smithi033 ceph-mon[31417]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:17 smithi152 ceph-mon[36015]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:17 smithi076 ceph-mon[35642]: pgmap v199: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:17.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 252246 5.39 MB/sec execute 194 sec latency 53.376 ms 2024-01-25T16:14:18.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 254297 5.39 MB/sec execute 195 sec latency 25.972 ms 2024-01-25T16:14:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:19 smithi033 ceph-mon[31417]: pgmap v200: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:19.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:19 smithi152 ceph-mon[36015]: pgmap v200: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:19 smithi076 ceph-mon[35642]: pgmap v200: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:19.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 256304 5.43 MB/sec execute 196 sec latency 14.852 ms 2024-01-25T16:14:20.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 258977 5.49 MB/sec execute 197 sec latency 15.130 ms 2024-01-25T16:14:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 -- 172.21.15.33:0/1867779573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 msgr2=0x7f3d380687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/1867779573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d380687d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c02bcd0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 -- 172.21.15.33:0/1867779573 shutdown_connections 2024-01-25T16:14:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/1867779573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38068d10 0x7f3d3810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/1867779573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d380687d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/1867779573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38106de0 0x7f3d381071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 -- 172.21.15.33:0/1867779573 >> 172.21.15.33:0/1867779573 conn(0x7f3d38076a40 msgr2=0x7f3d38076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 -- 172.21.15.33:0/1867779573 shutdown_connections 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.539+0000 7f3d3dd48640 1 -- 172.21.15.33:0/1867779573 wait complete. 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 Processor -- start 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 -- start start 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 0x7f3d381977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38106de0 0x7f3d3819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819c0d0 con 0x7f3d38068370 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819c240 con 0x7f3d38068d10 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d3dd48640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3819c400 con 0x7f3d38106de0 2024-01-25T16:14:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41160/0 (socket says 172.21.15.33:41160) 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d377fe640 1 -- 172.21.15.33:0/3743815035 learned_addr learned my addr 172.21.15.33:0/3743815035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 0x7f3d381977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.540+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38106de0 0x7f3d3819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 -- 172.21.15.33:0/3743815035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38106de0 msgr2=0x7f3d3819bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38106de0 0x7f3d3819bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 -- 172.21.15.33:0/3743815035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 msgr2=0x7f3d381977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 0x7f3d381977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 -- 172.21.15.33:0/3743815035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c009560 con 0x7f3d38068370 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 0x7f3d381977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d34ff9640 1 -- 172.21.15.33:0/3743815035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28009700 con 0x7f3d38068370 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d34ff9640 1 -- 172.21.15.33:0/3743815035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d2800cd80 con 0x7f3d38068370 2024-01-25T16:14:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3819c6e0 con 0x7f3d38068370 2024-01-25T16:14:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d34ff9640 1 -- 172.21.15.33:0/3743815035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28018860 con 0x7f3d38068370 2024-01-25T16:14:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.541+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381a3f60 con 0x7f3d38068370 2024-01-25T16:14:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.543+0000 7f3d34ff9640 1 -- 172.21.15.33:0/3743815035 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3d280189c0 con 0x7f3d38068370 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.543+0000 7f3d409ea740 1 Processor -- start 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 -- start start 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6e7cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6e6a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ddf070 con 0x560cc6e7c700 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6ddc760 con 0x560cc6e6a460 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d409ea740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6dd8050 con 0x560cc6e6ae20 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d34ff9640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 0x7f3d200a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6e7cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6e6a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d36ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d36ffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e845d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:37358/0 (socket says 172.21.15.33:37358) 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d36ffd640 1 -- 172.21.15.33:0/3939149340 learned_addr learned my addr 172.21.15.33:0/3939149340 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 0x7f3d200a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d34ff9640 1 -- 172.21.15.33:0/3743815035 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d28014070 con 0x7f3d38068370 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 -- 172.21.15.33:0/3939149340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 msgr2=0x560cc6e845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 -- 172.21.15.33:0/3939149340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 msgr2=0x560cc6e7cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6e7cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d377fe640 1 -- 172.21.15.33:0/3939149340 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc6dcfe70 con 0x560cc6e6a460 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.544+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6e7cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6e6a8e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d280ca930 tx=0x7f3d28099bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53983e5491e89e45 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 0x7f3d200a39b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x560cc6e78f40 tx=0x7f3d30004ab0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d1dffb640 1 -- 172.21.15.33:0/3939149340 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2809e130 con 0x560cc6e6a460 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d1dffb640 1 -- 172.21.15.33:0/3939149340 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d280a2370 con 0x560cc6e6a460 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d1dffb640 1 -- 172.21.15.33:0/3939149340 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280cf0d0 con 0x560cc6e6a460 2024-01-25T16:14:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 -- 172.21.15.33:0/3939149340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 msgr2=0x560cc6e6a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6e6a8e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d280ca930 tx=0x7f3d28099bd0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 -- 172.21.15.33:0/3939149340 shutdown_connections 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6e6a8e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3939149340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6e7cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.545+0000 7f3d409ea740 1 -- 172.21.15.33:0/3939149340 >> 172.21.15.33:0/3939149340 conn(0x560cc6e6cf20 msgr2=0x560cc6e6d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- 172.21.15.33:0/3939149340 shutdown_connections 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- 172.21.15.33:0/3939149340 wait complete. 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- start start 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 Processor -- start 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e81ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.556 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e7c700 0x560cc6e82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6d911e0 con 0x560cc6e6a460 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6d90e80 con 0x560cc6e7c700 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6d560c0 con 0x560cc6e6ae20 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d409ea740 10 client.0 _ll_register_callbacks cb 0x560cc6d7bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41174/0 (socket says 172.21.15.33:41174) 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d377fe640 1 -- 172.21.15.33:0/290699148 learned_addr learned my addr 172.21.15.33:0/290699148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d37fff640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e7c700 0x560cc6e82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.546+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e81ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 -- 172.21.15.33:0/290699148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 msgr2=0x560cc6e81ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e81ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 -- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e7c700 msgr2=0x560cc6e82410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.557 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e7c700 0x560cc6e82410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 -- 172.21.15.33:0/290699148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d381a3f60 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d377fe640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3d280a1e40 tx=0x7f3d28099d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d0c1d6bf89caba server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280a20e0 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d2809e130 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560cc6dd1040 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d280967a0 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d409ea740 10 client.15918 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560cc6dd66d0 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.547+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560cc6dd1040 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3d280b22f0 con 0x560cc6e6a460 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 10 client.15918 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:21.558 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 20 client.15918 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6e919a0 0x560cc6e93d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 4 client.15918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cc6dd0d60 con 0x560cc6e919a0 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6e919a0 0x560cc6e93d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6e919a0 0x560cc6e93d90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a1c77417d4356a5 server_cookie=9f2fa6f9663a5f73 in_seq=0 out_seq=0 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.548+0000 7f3d16ffd640 10 client.15918 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d16ffd640 1 -- 172.21.15.33:0/290699148 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cc6dd0d60 con 0x560cc6e919a0 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d16ffd640 10 client.15918 handle_command_reply: tid=0 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 client.15918 shutdown 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 msgr2=0x560cc6e81990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.559 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3d280a1e40 tx=0x7f3d28099d00 comp rx=0 tx=0).stop 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 shutdown_connections 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6e919a0 0x560cc6e93d90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e7c700 0x560cc6e82410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6e81ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d16ffd640 0 client.15918 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 --2- 172.21.15.33:0/290699148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e81990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 >> 172.21.15.33:0/290699148 conn(0x560cc6e6cf20 msgr2=0x560cc6e7ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 shutdown_connections 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.549+0000 7f3d409ea740 1 -- 172.21.15.33:0/290699148 wait complete. 2024-01-25T16:14:21.560 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.550+0000 7f3d409ea740 20 client.15918 trim_cache size 0 max 16384 2024-01-25T16:14:21.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:21 smithi033 ceph-mon[31417]: pgmap v201: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 Processor -- start 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 -- start start 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6ae20 0x560cc6e88c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 0x560cc6e891d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.570 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6e98ee0 con 0x560cc6e6a460 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6e7fee0 con 0x560cc6e6ae20 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d409ea740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6e7ce10 con 0x560cc6e7c700 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.563+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41190/0 (socket says 172.21.15.33:41190) 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 -- 172.21.15.33:0/3882408068 learned_addr learned my addr 172.21.15.33:0/3882408068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6ae20 0x560cc6e88c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 0x560cc6e891d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 -- 172.21.15.33:0/3882408068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 msgr2=0x560cc6e891d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 0x560cc6e891d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 -- 172.21.15.33:0/3882408068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6ae20 msgr2=0x560cc6e88c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6ae20 0x560cc6e88c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 -- 172.21.15.33:0/3882408068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cc6d73060 con 0x560cc6e6a460 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 0x560cc6e891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d280983e0 tx=0x7f3d280b8250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaa1605b072e5a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.571 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d1d7fa640 1 -- 172.21.15.33:0/3882408068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2809f200 con 0x560cc6e6a460 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d1d7fa640 1 -- 172.21.15.33:0/3882408068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d280967a0 con 0x560cc6e6a460 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.564+0000 7f3d1d7fa640 1 -- 172.21.15.33:0/3882408068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d28098ff0 con 0x560cc6e6a460 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- 172.21.15.33:0/3882408068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 msgr2=0x560cc6e7a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3d280983e0 tx=0x7f3d280b8250 comp rx=0 tx=0).stop 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- 172.21.15.33:0/3882408068 shutdown_connections 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e7c700 0x560cc6e891d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6ae20 0x560cc6e88c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3882408068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e6a460 0x560cc6e7a1e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- 172.21.15.33:0/3882408068 >> 172.21.15.33:0/3882408068 conn(0x560cc6e6cf20 msgr2=0x560cc6e7ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- 172.21.15.33:0/3882408068 shutdown_connections 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- 172.21.15.33:0/3882408068 wait complete. 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- start start 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 Processor -- start 2024-01-25T16:14:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6ee1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6ee1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6edc370 con 0x560cc6e7c700 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6edc4e0 con 0x560cc6e6a460 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.565+0000 7f3d409ea740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cc6edc6a0 con 0x560cc6e6ae20 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d409ea740 10 client.0 _ll_register_callbacks cb 0x7f3d28095d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41192/0 (socket says 172.21.15.33:41192) 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 -- 172.21.15.33:0/3966231587 learned_addr learned my addr 172.21.15.33:0/3966231587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6ee1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d36ffd640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6ee1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 -- 172.21.15.33:0/3966231587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 msgr2=0x560cc6ee1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6ee1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 -- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 msgr2=0x560cc6ee1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6ee1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 -- 172.21.15.33:0/3966231587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cc6dd66d0 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d37fff640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000efc0 tx=0x7f3d30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5959f234dd848b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30003e00 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d300047c0 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d280a2370 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d30010460 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d409ea740 10 client.15924 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.566+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560cc6d98a80 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d280a2370 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3d30034690 con 0x560cc6e7c700 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 10 client.15924 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 20 client.15924 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6ee9430 0x560cc6eeb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 4 client.15924 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560cc6ad7600 con 0x560cc6ee9430 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6ee9430 0x560cc6eeb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d377fe640 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6ee9430 0x560cc6eeb850 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce1c5ad216bae67 server_cookie=aab9b2c4e02b3842 in_seq=0 out_seq=0 2024-01-25T16:14:21.575 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.567+0000 7f3d16ffd640 10 client.15924 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:21 smithi152 ceph-mon[36015]: pgmap v201: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:21 smithi076 ceph-mon[35642]: pgmap v201: 97 pgs: 97 active+clean; 73 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.9 MiB/s wr, 355 op/s 2024-01-25T16:14:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d16ffd640 1 -- 172.21.15.33:0/3966231587 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560cc6ad7600 con 0x560cc6ee9430 2024-01-25T16:14:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d16ffd640 10 client.15924 handle_command_reply: tid=0 2024-01-25T16:14:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 client.15924 shutdown 2024-01-25T16:14:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 msgr2=0x560cc6edbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3d3000efc0 tx=0x7f3d30004a40 comp rx=0 tx=0).stop 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 shutdown_connections 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x560cc6ee9430 0x560cc6eeb850 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cc6e7c700 0x560cc6edbd60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x560cc6e6ae20 0x560cc6ee1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 --2- 172.21.15.33:0/3966231587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560cc6e6a460 0x560cc6ee1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d16ffd640 0 client.15924 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 >> 172.21.15.33:0/3966231587 conn(0x560cc6e6cf20 msgr2=0x560cc6e7ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 shutdown_connections 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.710+0000 7f3d409ea740 1 -- 172.21.15.33:0/3966231587 wait complete. 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d409ea740 20 client.15924 trim_cache size 0 max 16384 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:14:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:14:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:14:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 msgr2=0x7f3d200a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 0x7f3d200a39b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x560cc6e78f40 tx=0x7f3d30004ab0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 msgr2=0x7f3d38197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800e9f0 tx=0x7f3d2800eeb0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 shutdown_connections 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3d200a14f0 0x7f3d200a39b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3d38106de0 0x7f3d3819bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3d38068d10 0x7f3d381977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 --2- 172.21.15.33:0/3743815035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d38068370 0x7f3d38197260 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 >> 172.21.15.33:0/3743815035 conn(0x7f3d38076a40 msgr2=0x7f3d380fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 shutdown_connections 2024-01-25T16:14:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:21.711+0000 7f3d3dd48640 1 -- 172.21.15.33:0/3743815035 wait complete. 2024-01-25T16:14:21.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 260637 5.50 MB/sec execute 198 sec latency 74.317 ms 2024-01-25T16:14:22.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 262808 5.54 MB/sec execute 199 sec latency 58.359 ms 2024-01-25T16:14:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:23 smithi033 ceph-mon[31417]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-01-25T16:14:23.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:23 smithi152 ceph-mon[36015]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-01-25T16:14:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:23 smithi076 ceph-mon[35642]: pgmap v202: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 405 op/s 2024-01-25T16:14:23.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 264872 5.55 MB/sec execute 200 sec latency 17.002 ms 2024-01-25T16:14:24.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 266863 5.58 MB/sec execute 201 sec latency 19.282 ms 2024-01-25T16:14:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:25 smithi033 ceph-mon[31417]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-01-25T16:14:25.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:25 smithi152 ceph-mon[36015]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-01-25T16:14:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:25 smithi076 ceph-mon[35642]: pgmap v203: 97 pgs: 97 active+clean; 80 MiB data, 1012 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.2 MiB/s wr, 217 op/s 2024-01-25T16:14:25.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 269058 5.61 MB/sec execute 202 sec latency 63.933 ms 2024-01-25T16:14:26.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 271095 5.64 MB/sec execute 203 sec latency 14.774 ms 2024-01-25T16:14:27.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:27 smithi033 ceph-mon[31417]: pgmap v204: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 410 op/s 2024-01-25T16:14:27.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:27 smithi152 ceph-mon[36015]: pgmap v204: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 410 op/s 2024-01-25T16:14:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:27 smithi076 ceph-mon[35642]: pgmap v204: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.7 MiB/s wr, 410 op/s 2024-01-25T16:14:27.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 273376 5.68 MB/sec execute 204 sec latency 41.261 ms 2024-01-25T16:14:28.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 275067 5.68 MB/sec execute 205 sec latency 34.213 ms 2024-01-25T16:14:29.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:29 smithi076 ceph-mon[35642]: pgmap v205: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:29 smithi033 ceph-mon[31417]: pgmap v205: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:29.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:29 smithi152 ceph-mon[36015]: pgmap v205: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:29.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 276812 5.72 MB/sec execute 206 sec latency 350.123 ms 2024-01-25T16:14:30.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 278893 5.72 MB/sec execute 207 sec latency 14.994 ms 2024-01-25T16:14:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:31 smithi033 ceph-mon[31417]: pgmap v206: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:31.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:31 smithi152 ceph-mon[36015]: pgmap v206: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:31 smithi076 ceph-mon[35642]: pgmap v206: 97 pgs: 97 active+clean; 70 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 301 op/s 2024-01-25T16:14:31.776 INFO:tasks.workunit.client.0.smithi033.stdout: 1 280984 5.76 MB/sec execute 208 sec latency 17.617 ms 2024-01-25T16:14:32.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 283217 5.79 MB/sec execute 209 sec latency 102.377 ms 2024-01-25T16:14:33.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:33 smithi033 ceph-mon[31417]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-01-25T16:14:33.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:33 smithi152 ceph-mon[36015]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-01-25T16:14:33.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:33 smithi076 ceph-mon[35642]: pgmap v207: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.6 MiB/s wr, 403 op/s 2024-01-25T16:14:33.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 285220 5.82 MB/sec execute 210 sec latency 19.309 ms 2024-01-25T16:14:34.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287592 5.86 MB/sec execute 211 sec latency 21.024 ms 2024-01-25T16:14:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.801+0000 7f40f5a83640 1 -- 172.21.15.33:0/3815843455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f0104b90 msgr2=0x7f40f0104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.801+0000 7f40f5a83640 1 --2- 172.21.15.33:0/3815843455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f0104b90 0x7f40f0104f70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009890 tx=0x7f40e002bb70 comp rx=0 tx=0).stop 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 -- 172.21.15.33:0/3815843455 shutdown_connections 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 --2- 172.21.15.33:0/3815843455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 --2- 172.21.15.33:0/3815843455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f00ffe40 0x7f40f01002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 --2- 172.21.15.33:0/3815843455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f0104b90 0x7f40f0104f70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 -- 172.21.15.33:0/3815843455 >> 172.21.15.33:0/3815843455 conn(0x7f40f0076a40 msgr2=0x7f40f0076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 -- 172.21.15.33:0/3815843455 shutdown_connections 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 -- 172.21.15.33:0/3815843455 wait complete. 2024-01-25T16:14:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 Processor -- start 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.802+0000 7f40f5a83640 1 -- start start 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f00ffe40 0x7f40f01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f0198e90 con 0x7f40f0104b90 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01a3690 con 0x7f40f00ffe40 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40f5a83640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f01a37d0 con 0x7f40f0100800 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f00ffe40 0x7f40f01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44906/0 (socket says 172.21.15.33:44906) 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40effff640 1 -- 172.21.15.33:0/2611858085 learned_addr learned my addr 172.21.15.33:0/2611858085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 -- 172.21.15.33:0/2611858085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f00ffe40 msgr2=0x7f40f01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f00ffe40 0x7f40f01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 -- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 msgr2=0x7f40f01a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 -- 172.21.15.33:0/2611858085 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0009490 con 0x7f40f0100800 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40effff640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40eeffd640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f0198610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40e400e790 tx=0x7f40e400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.803+0000 7f40ecff9640 1 -- 172.21.15.33:0/2611858085 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e400ccc0 con 0x7f40f0100800 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.804+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f01a39d0 con 0x7f40f0100800 2024-01-25T16:14:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.804+0000 7f40ecff9640 1 -- 172.21.15.33:0/2611858085 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e400eea0 con 0x7f40f0100800 2024-01-25T16:14:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.804+0000 7f40ecff9640 1 -- 172.21.15.33:0/2611858085 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e4018680 con 0x7f40f0100800 2024-01-25T16:14:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.804+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f01a3ed0 con 0x7f40f0100800 2024-01-25T16:14:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40ecff9640 1 -- 172.21.15.33:0/2611858085 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f40e4018800 con 0x7f40f0100800 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 Processor -- start 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 -- start start 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d411017070 con 0x55d4110b4700 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d411014760 con 0x55d4110a2460 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40ecff9640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 0x7f40d80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.807+0000 7f40f8645740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d411010050 con 0x55d4110a2e20 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44916/0 (socket says 172.21.15.33:44916) 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40effff640 1 -- 172.21.15.33:0/1892294436 learned_addr learned my addr 172.21.15.33:0/1892294436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ecff9640 1 -- 172.21.15.33:0/2611858085 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40e4014070 con 0x7f40f0100800 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40effff640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 0x7f40d80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 -- 172.21.15.33:0/1892294436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 msgr2=0x55d4110bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 -- 172.21.15.33:0/1892294436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 msgr2=0x55d4110b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110b4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 -- 172.21.15.33:0/1892294436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d411007e70 con 0x55d4110a2460 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110a28e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40e002cf50 tx=0x7f40e002cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb483ac6b91a0e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.808+0000 7f40effff640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 0x7f40d80a39b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8004ab0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40d5ffb640 1 -- 172.21.15.33:0/1892294436 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0034cd0 con 0x55d4110a2460 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40d5ffb640 1 -- 172.21.15.33:0/1892294436 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e0004500 con 0x55d4110a2460 2024-01-25T16:14:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40d5ffb640 1 -- 172.21.15.33:0/1892294436 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e002c710 con 0x55d4110a2460 2024-01-25T16:14:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 -- 172.21.15.33:0/1892294436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 msgr2=0x55d4110a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110a28e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40e002cf50 tx=0x7f40e002cea0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 -- 172.21.15.33:0/1892294436 shutdown_connections 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110a28e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 --2- 172.21.15.33:0/1892294436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 -- 172.21.15.33:0/1892294436 >> 172.21.15.33:0/1892294436 conn(0x55d4110a4f20 msgr2=0x55d4110a5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 -- 172.21.15.33:0/1892294436 shutdown_connections 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.809+0000 7f40f8645740 1 -- 172.21.15.33:0/1892294436 wait complete. 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 -- start start 2024-01-25T16:14:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 Processor -- start 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110ba0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d410fc91e0 con 0x55d4110b4700 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d410fc8e80 con 0x55d4110a2460 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d410f8e0c0 con 0x55d4110a2e20 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40f8645740 10 client.0 _ll_register_callbacks cb 0x55d410fb3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110ba0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110bc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:36300/0 (socket says 172.21.15.33:36300) 2024-01-25T16:14:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.810+0000 7f40ef7fe640 1 -- 172.21.15.33:0/3104392615 learned_addr learned my addr 172.21.15.33:0/3104392615 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 -- 172.21.15.33:0/3104392615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 msgr2=0x55d4110b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 -- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 msgr2=0x55d4110bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 -- 172.21.15.33:0/3104392615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f010d3d0 con 0x55d4110b4700 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40effff640 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110ba0d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f40e80099f0 tx=0x7f40e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc08a31a45f989d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8022950 con 0x55d4110b4700 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40e8022ab0 con 0x55d4110b4700 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d411008d60 con 0x55d4110b4700 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e8022c70 con 0x55d4110b4700 2024-01-25T16:14:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40f8645740 10 client.15936 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55d41100e6d0 con 0x55d4110b4700 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d411008d60 con 0x55d4110b4700 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.811+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40e802a3b0 con 0x55d4110b4700 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 10 client.15936 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 10 client.15936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 20 client.15936 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d4110c9ce0 0x55d4110cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 4 client.15936 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d410d0f600 con 0x55d4110c9ce0 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d4110c9ce0 0x55d4110cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40ceffd640 10 client.15936 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.812+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d4110c9ce0 0x55d4110cc0d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=954d4457b2512e21 server_cookie=aa36da7c45ab57a1 in_seq=0 out_seq=0 2024-01-25T16:14:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40ceffd640 1 -- 172.21.15.33:0/3104392615 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d410d0f600 con 0x55d4110c9ce0 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40ceffd640 10 client.15936 handle_command_reply: tid=0 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 client.15936 shutdown 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 msgr2=0x55d4110ba0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110ba0d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f40e80099f0 tx=0x7f40e8004a40 comp rx=0 tx=0).stop 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 shutdown_connections 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d4110c9ce0 0x55d4110cc0d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110ba0d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40ceffd640 0 client.15936 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 --2- 172.21.15.33:0/3104392615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 >> 172.21.15.33:0/3104392615 conn(0x55d4110a4f20 msgr2=0x55d4110b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 shutdown_connections 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 1 -- 172.21.15.33:0/3104392615 wait complete. 2024-01-25T16:14:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.813+0000 7f40f8645740 20 client.15936 trim_cache size 0 max 16384 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 Processor -- start 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 -- start start 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 0x55d4110d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110b4700 0x55d4110c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4110a3360 con 0x55d4110a2e20 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41109e2e0 con 0x55d4110b4700 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40f8645740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d41109cf30 con 0x55d4110a2460 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.827+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.831 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44926/0 (socket says 172.21.15.33:44926) 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 0x55d4110d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 -- 172.21.15.33:0/1739764066 learned_addr learned my addr 172.21.15.33:0/1739764066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40effff640 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110b4700 0x55d4110c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 -- 172.21.15.33:0/1739764066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 msgr2=0x55d4110d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 0x55d4110d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 -- 172.21.15.33:0/1739764066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110b4700 msgr2=0x55d4110c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110b4700 0x55d4110c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 -- 172.21.15.33:0/1739764066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d410fab060 con 0x55d4110a2e20 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 0x55d4110d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40eeffd640 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f40e40ca8d0 tx=0x7f40e4025310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436e8afb0f13e952 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40d57fa640 1 -- 172.21.15.33:0/1739764066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e400ce20 con 0x55d4110a2e20 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40d57fa640 1 -- 172.21.15.33:0/1739764066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e4099fd0 con 0x55d4110a2e20 2024-01-25T16:14:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.828+0000 7f40d57fa640 1 -- 172.21.15.33:0/1739764066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e401dcf0 con 0x55d4110a2e20 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- 172.21.15.33:0/1739764066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 msgr2=0x55d4110c0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f40e40ca8d0 tx=0x7f40e4025310 comp rx=0 tx=0).stop 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- 172.21.15.33:0/1739764066 shutdown_connections 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110b4700 0x55d4110c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110a2e20 0x55d4110c0c90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 --2- 172.21.15.33:0/1739764066 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2460 0x55d4110d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.836 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- 172.21.15.33:0/1739764066 >> 172.21.15.33:0/1739764066 conn(0x55d4110a4f20 msgr2=0x55d4110a11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- 172.21.15.33:0/1739764066 shutdown_connections 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- 172.21.15.33:0/1739764066 wait complete. 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 -- start start 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.829+0000 7f40f8645740 1 Processor -- start 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4110bfcf0 con 0x55d4110b4700 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4110bfe60 con 0x55d4110a2460 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4110c0020 con 0x55d4110a2e20 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40f8645740 10 client.0 _ll_register_callbacks cb 0x7f40e4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2024-01-25T16:14:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 -- 172.21.15.33:0/786956317 learned_addr learned my addr 172.21.15.33:0/786956317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40effff640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40eeffd640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 -- 172.21.15.33:0/786956317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 msgr2=0x55d4110bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 -- 172.21.15.33:0/786956317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 msgr2=0x55d4110bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110bf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 -- 172.21.15.33:0/786956317 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d411007e70 con 0x55d4110a2460 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ef7fe640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40e002c180 tx=0x7f40e0009830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95ddd663d709086 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.830+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0051de0 con 0x55d4110a2460 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d411009040 con 0x55d4110a2460 2024-01-25T16:14:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f40e004e040 con 0x55d4110a2460 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40e0034440 con 0x55d4110a2460 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 10 client.25460 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f40f01a3ed0 con 0x55d4110a2460 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d411009040 con 0x55d4110a2460 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f40e00514b0 con 0x55d4110a2460 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 10 client.25460 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 10 client.25460 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 20 client.25460 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d411121380 0x55d4111237a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 4 client.25460 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d410d62c50 con 0x55d411121380 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.831+0000 7f40eeffd640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d411121380 0x55d4111237a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.832+0000 7f40eeffd640 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d411121380 0x55d4111237a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa8ba6bec2d80fe server_cookie=d56a9aa0bdf480f1 in_seq=0 out_seq=0 2024-01-25T16:14:34.839 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:34.832+0000 7f40ceffd640 10 client.25460 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.003+0000 7f40ceffd640 1 -- 172.21.15.33:0/786956317 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d410d62c50 con 0x55d411121380 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.003+0000 7f40ceffd640 10 client.25460 handle_command_reply: tid=0 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 client.25460 shutdown 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 msgr2=0x55d4110c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40e002c180 tx=0x7f40e0009830 comp rx=0 tx=0).stop 2024-01-25T16:14:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 shutdown_connections 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d411121380 0x55d4111237a0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4110b4700 0x55d4110bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d4110a2e20 0x55d4110bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 --2- 172.21.15.33:0/786956317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d4110a2460 0x55d4110c0d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 >> 172.21.15.33:0/786956317 conn(0x55d4110a4f20 msgr2=0x55d4110d0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40ceffd640 0 client.25460 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 shutdown_connections 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 1 -- 172.21.15.33:0/786956317 wait complete. 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.004+0000 7f40f8645740 20 client.25460 trim_cache size 0 max 16384 2024-01-25T16:14:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:14:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:14:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:14:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:14:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 msgr2=0x7f40d80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 0x7f40d80a39b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009fd0 tx=0x7f40e8004ab0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 msgr2=0x7f40f0198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f0198610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40e400e790 tx=0x7f40e400ec50 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 shutdown_connections 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f40d80a14f0 0x7f40d80a39b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f40f0104b90 0x7f40f01a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f40f0100800 0x7f40f0198610 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 --2- 172.21.15.33:0/2611858085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40f00ffe40 0x7f40f01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 >> 172.21.15.33:0/2611858085 conn(0x7f40f0076a40 msgr2=0x7f40f00fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 shutdown_connections 2024-01-25T16:14:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:35.005+0000 7f40f5a83640 1 -- 172.21.15.33:0/2611858085 wait complete. 2024-01-25T16:14:35.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:35 smithi033 ceph-mon[31417]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-01-25T16:14:35.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:35 smithi152 ceph-mon[36015]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-01-25T16:14:35.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:35 smithi076 ceph-mon[35642]: pgmap v208: 97 pgs: 97 active+clean; 76 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.0 MiB/s wr, 294 op/s 2024-01-25T16:14:35.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 289018 5.86 MB/sec execute 212 sec latency 101.976 ms 2024-01-25T16:14:36.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:36 smithi033 ceph-mon[31417]: pgmap v209: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-01-25T16:14:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:36 smithi152 ceph-mon[36015]: pgmap v209: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-01-25T16:14:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:36 smithi076 ceph-mon[35642]: pgmap v209: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-01-25T16:14:36.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 291368 5.89 MB/sec execute 213 sec latency 15.023 ms 2024-01-25T16:14:37.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 293360 5.90 MB/sec execute 214 sec latency 88.830 ms 2024-01-25T16:14:38.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:38 smithi033 ceph-mon[31417]: pgmap v210: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:38.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:38 smithi152 ceph-mon[36015]: pgmap v210: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:38.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:38 smithi076 ceph-mon[35642]: pgmap v210: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:38.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 295372 5.93 MB/sec execute 215 sec latency 19.466 ms 2024-01-25T16:14:39.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 298046 5.98 MB/sec execute 216 sec latency 14.860 ms 2024-01-25T16:14:40.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 299532 5.98 MB/sec execute 217 sec latency 57.966 ms 2024-01-25T16:14:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:41 smithi033 ceph-mon[31417]: pgmap v211: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:41.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:41 smithi152 ceph-mon[36015]: pgmap v211: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:41.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:41 smithi076 ceph-mon[35642]: pgmap v211: 97 pgs: 97 active+clean; 66 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-01-25T16:14:41.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 301902 6.02 MB/sec execute 218 sec latency 14.061 ms 2024-01-25T16:14:42.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 303891 6.03 MB/sec execute 219 sec latency 49.276 ms 2024-01-25T16:14:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:43 smithi033 ceph-mon[31417]: pgmap v212: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-01-25T16:14:43.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:43 smithi152 ceph-mon[36015]: pgmap v212: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-01-25T16:14:43.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:43 smithi076 ceph-mon[35642]: pgmap v212: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 402 op/s 2024-01-25T16:14:43.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 305846 6.05 MB/sec execute 220 sec latency 15.682 ms 2024-01-25T16:14:44.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 308589 6.10 MB/sec execute 221 sec latency 16.053 ms 2024-01-25T16:14:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:45 smithi033 ceph-mon[31417]: pgmap v213: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2024-01-25T16:14:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:45 smithi152 ceph-mon[36015]: pgmap v213: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2024-01-25T16:14:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:45 smithi076 ceph-mon[35642]: pgmap v213: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 300 op/s 2024-01-25T16:14:45.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 310222 6.11 MB/sec execute 222 sec latency 62.501 ms 2024-01-25T16:14:46.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 312565 6.14 MB/sec execute 223 sec latency 16.232 ms 2024-01-25T16:14:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:47 smithi033 ceph-mon[31417]: pgmap v214: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-01-25T16:14:47.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:47 smithi152 ceph-mon[36015]: pgmap v214: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-01-25T16:14:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:47 smithi076 ceph-mon[35642]: pgmap v214: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-01-25T16:14:47.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 314563 6.15 MB/sec execute 224 sec latency 45.831 ms 2024-01-25T16:14:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.100+0000 7f1364236640 1 -- 172.21.15.33:0/208986805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c0ffe20 msgr2=0x7f135c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.100+0000 7f1364236640 1 --2- 172.21.15.33:0/208986805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c0ffe20 0x7f135c100200 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1358009930 tx=0x7f135802bc30 comp rx=0 tx=0).stop 2024-01-25T16:14:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 -- 172.21.15.33:0/208986805 shutdown_connections 2024-01-25T16:14:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 --2- 172.21.15.33:0/208986805 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c104ef0 0x7f135c10cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 --2- 172.21.15.33:0/208986805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c1007d0 0x7f135c104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 --2- 172.21.15.33:0/208986805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c0ffe20 0x7f135c100200 secure :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1358009930 tx=0x7f135802bc30 comp rx=0 tx=0).stop 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 -- 172.21.15.33:0/208986805 >> 172.21.15.33:0/208986805 conn(0x7f135c076d10 msgr2=0x7f135c077120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 -- 172.21.15.33:0/208986805 shutdown_connections 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.101+0000 7f1364236640 1 -- 172.21.15.33:0/208986805 wait complete. 2024-01-25T16:14:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.102+0000 7f1364236640 1 Processor -- start 2024-01-25T16:14:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 -- start start 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 0x7f135c194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c195510 0x7f135c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c199bd0 con 0x7f135c104ef0 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c199d40 con 0x7f135c195510 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1364236640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c199f00 con 0x7f135c19c490 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52398/0 (socket says 172.21.15.33:52398) 2024-01-25T16:14:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f1361fd2640 1 -- 172.21.15.33:0/3865242008 learned_addr learned my addr 172.21.15.33:0/3865242008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f13617d1640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 0x7f135c194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.103+0000 7f13627d3640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c195510 0x7f135c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 -- 172.21.15.33:0/3865242008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c195510 msgr2=0x7f135c1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c195510 0x7f135c1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 -- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 msgr2=0x7f135c19bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 -- 172.21.15.33:0/3865242008 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358009560 con 0x7f135c19c490 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f13617d1640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 0x7f135c194fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e9c0 tx=0x7f134c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1352ffd640 1 -- 172.21.15.33:0/3865242008 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c00ccc0 con 0x7f135c19c490 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c19a1e0 con 0x7f135c19c490 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1352ffd640 1 -- 172.21.15.33:0/3865242008 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134c00ce20 con 0x7f135c19c490 2024-01-25T16:14:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c19a740 con 0x7f135c19c490 2024-01-25T16:14:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.104+0000 7f1352ffd640 1 -- 172.21.15.33:0/3865242008 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0187e0 con 0x7f135c19c490 2024-01-25T16:14:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.108+0000 7f1352ffd640 1 -- 172.21.15.33:0/3865242008 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f134c018940 con 0x7f135c19c490 2024-01-25T16:14:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.109+0000 7f1352ffd640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 0x7f13480a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.109+0000 7f1352ffd640 1 -- 172.21.15.33:0/3865242008 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f134c014070 con 0x7f135c19c490 2024-01-25T16:14:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.109+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 0x7f13480a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.109+0000 7f1366dfd740 1 Processor -- start 2024-01-25T16:14:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.109+0000 7f1366dfd740 1 -- start start 2024-01-25T16:14:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e710240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e712b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e676070 con 0x55723e710780 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e673760 con 0x55723e707090 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1366dfd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e66f050 con 0x55723e7079b0 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 0x7f13480a3ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f1358005a20 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f13617d1640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e710240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:56920/0 (socket says 172.21.15.33:56920) 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 -- 172.21.15.33:0/1267409191 learned_addr learned my addr 172.21.15.33:0/1267409191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f13627d3640 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e712b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 -- 172.21.15.33:0/1267409191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 msgr2=0x55723e710240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e710240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 -- 172.21.15.33:0/1267409191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 msgr2=0x55723e712b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e712b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f1361fd2640 1 -- 172.21.15.33:0/1267409191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723e666e70 con 0x55723e707090 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.110+0000 7f13627d3640 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e712b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:14:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f135802c240 tx=0x7f135803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db00251a610aba4a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1347fff640 1 -- 172.21.15.33:0/1267409191 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135802cb40 con 0x55723e707090 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1347fff640 1 -- 172.21.15.33:0/1267409191 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1358004500 con 0x55723e707090 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1347fff640 1 -- 172.21.15.33:0/1267409191 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358004dd0 con 0x55723e707090 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1366dfd740 1 -- 172.21.15.33:0/1267409191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 msgr2=0x55723e707470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.111+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f135802c240 tx=0x7f135803f000 comp rx=0 tx=0).stop 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 -- 172.21.15.33:0/1267409191 shutdown_connections 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e712b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e710240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1267409191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e707470 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 -- 172.21.15.33:0/1267409191 >> 172.21.15.33:0/1267409191 conn(0x55723e7019b0 msgr2=0x55723e701d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 -- 172.21.15.33:0/1267409191 shutdown_connections 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 -- 172.21.15.33:0/1267409191 wait complete. 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 -- start start 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 Processor -- start 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 0x55723e70d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.112+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e70e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e6281e0 con 0x55723e710780 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e627e80 con 0x55723e7079b0 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e5ed0c0 con 0x55723e707090 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 10 client.0 _ll_register_callbacks cb 0x55723e612ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 0x55723e70d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:56936/0 (socket says 172.21.15.33:56936) 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 -- 172.21.15.33:0/3642188412 learned_addr learned my addr 172.21.15.33:0/3642188412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13627d3640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e70e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 -- 172.21.15.33:0/3642188412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 msgr2=0x55723e70d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 0x55723e70d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 -- 172.21.15.33:0/3642188412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 msgr2=0x55723e70e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e70e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 -- 172.21.15.33:0/3642188412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c10d2d0 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 0x55723e70d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13617d1640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f134c0973d0 tx=0x7f134c09ab60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8175214e5be8dae1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13457fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c0b2070 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55723e667d60 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f134c0a2d70 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f134c0a2f30 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 10 client.25472 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.113+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55723e66d6d0 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f134c07d020 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f134c096da0 con 0x55723e7079b0 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 10 client.25472 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 20 client.25472 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e720540 0x55723e722960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 4 client.25472 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:14:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55723e36e600 con 0x55723e720540 2024-01-25T16:14:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.114+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e720540 0x55723e722960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e720540 0x55723e722960 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa4c690953d8b379 server_cookie=7530d14a5b375f7d in_seq=0 out_seq=0 2024-01-25T16:14:48.122 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f13457fa640 10 client.25472 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f13457fa640 1 -- 172.21.15.33:0/3642188412 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55723e36e600 con 0x55723e720540 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f13457fa640 10 client.25472 handle_command_reply: tid=0 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f1366dfd740 1 client.25472 shutdown 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 msgr2=0x55723e70db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.115+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f134c0973d0 tx=0x7f134c09ab60 comp rx=0 tx=0).stop 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 shutdown_connections 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e720540 0x55723e722960 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e70e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e7079b0 0x55723e70db80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3642188412 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e707090 0x55723e70d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f13457fa640 0 client.25472 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 >> 172.21.15.33:0/3642188412 conn(0x55723e7019b0 msgr2=0x55723e703700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 shutdown_connections 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 1 -- 172.21.15.33:0/3642188412 wait complete. 2024-01-25T16:14:48.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.116+0000 7f1366dfd740 20 client.25472 trim_cache size 0 max 16384 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.130+0000 7f1366dfd740 1 Processor -- start 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 -- start start 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e70a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 0x55723e70ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e712ba0 con 0x55723e707090 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e7066a0 con 0x55723e710780 2024-01-25T16:14:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1366dfd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e70eab0 con 0x55723e7079b0 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f13627d3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 0x55723e70ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52428/0 (socket says 172.21.15.33:52428) 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 -- 172.21.15.33:0/3015812314 learned_addr learned my addr 172.21.15.33:0/3015812314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f13617d1640 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e70a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 -- 172.21.15.33:0/3015812314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 msgr2=0x55723e70a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e70a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 -- 172.21.15.33:0/3015812314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 msgr2=0x55723e70ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 0x55723e70ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 -- 172.21.15.33:0/3015812314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55723e60a060 con 0x55723e707090 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f13627d3640 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 0x55723e70ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f1361fd2640 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f135802c210 tx=0x7f1358042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bef9cb5907beab server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.131+0000 7f13477fe640 1 -- 172.21.15.33:0/3015812314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135805eb70 con 0x55723e707090 2024-01-25T16:14:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f13477fe640 1 -- 172.21.15.33:0/3015812314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13580044c0 con 0x55723e707090 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f13477fe640 1 -- 172.21.15.33:0/3015812314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135805e5e0 con 0x55723e707090 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 -- 172.21.15.33:0/3015812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 msgr2=0x55723e70fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f135802c210 tx=0x7f1358042040 comp rx=0 tx=0).stop 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 -- 172.21.15.33:0/3015812314 shutdown_connections 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e710780 0x55723e70ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e70a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 --2- 172.21.15.33:0/3015812314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e707090 0x55723e70fd70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 -- 172.21.15.33:0/3015812314 >> 172.21.15.33:0/3015812314 conn(0x55723e7019b0 msgr2=0x55723e70bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 -- 172.21.15.33:0/3015812314 shutdown_connections 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.132+0000 7f1366dfd740 1 -- 172.21.15.33:0/3015812314 wait complete. 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 -- start start 2024-01-25T16:14:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 Processor -- start 2024-01-25T16:14:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e715c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e716950 con 0x55723e710780 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e716ac0 con 0x55723e707090 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55723e776a30 con 0x55723e7079b0 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1366dfd740 10 client.0 _ll_register_callbacks cb 0x7f13540093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f13627d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1361fd2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:56948/0 (socket says 172.21.15.33:56948) 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f1361fd2640 1 -- 172.21.15.33:0/1955244055 learned_addr learned my addr 172.21.15.33:0/1955244055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f13627d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52432/0 (socket says 172.21.15.33:52432) 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.133+0000 7f13617d1640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e715c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 -- 172.21.15.33:0/1955244055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 msgr2=0x55723e715c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e715c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 -- 172.21.15.33:0/1955244055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 msgr2=0x55723e716280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 -- 172.21.15.33:0/1955244055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1354009c90 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13617d1640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e715c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13627d3640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1361fd2640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1358009900 tx=0x7f135802c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be9dc691d67980 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13457fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135806b070 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1358047050 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55723e668040 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1358004000 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1366dfd740 10 client.25478 fetch_fsmap learned FSMap version 26 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55723e666e70 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.134+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55723e668040 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f135806e020 con 0x55723e707090 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 10 client.25478 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 10 client.25478 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 20 client.25478 populate_metadata read hostname 'smithi033' 2024-01-25T16:14:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e777df0 0x55723e77a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:14:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 4 client.25478 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:14:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55723e3c1c50 con 0x55723e777df0 2024-01-25T16:14:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f13617d1640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e777df0 0x55723e77a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:14:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f13457fa640 10 client.25478 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.135+0000 7f13617d1640 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e777df0 0x55723e77a1e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3022fcb7d16fe9dd server_cookie=ea94c7f6f158f528 in_seq=0 out_seq=0 2024-01-25T16:14:48.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f13457fa640 1 -- 172.21.15.33:0/1955244055 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55723e3c1c50 con 0x55723e777df0 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f13457fa640 10 client.25478 handle_command_reply: tid=0 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 client.25478 shutdown 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 msgr2=0x55723e7178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1358009900 tx=0x7f135802c800 comp rx=0 tx=0).stop 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 shutdown_connections 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55723e777df0 0x55723e77a1e0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55723e710780 0x55723e716280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55723e7079b0 0x55723e715c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 --2- 172.21.15.33:0/1955244055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55723e707090 0x55723e7178b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 >> 172.21.15.33:0/1955244055 conn(0x55723e7019b0 msgr2=0x55723e70de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f13457fa640 0 client.25478 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 shutdown_connections 2024-01-25T16:14:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.253+0000 7f1366dfd740 1 -- 172.21.15.33:0/1955244055 wait complete. 2024-01-25T16:14:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1366dfd740 20 client.25478 trim_cache size 0 max 16384 2024-01-25T16:14:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:14:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:14:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:14:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 msgr2=0x7f13480a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 0x7f13480a3ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f1358005a20 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 msgr2=0x7f135c194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 0x7f135c194fd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f134c00e9c0 tx=0x7f134c00ee80 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 shutdown_connections 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13480a1610 0x7f13480a3ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f135c195510 0x7f135c1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f135c19c490 0x7f135c194fd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 --2- 172.21.15.33:0/3865242008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f135c104ef0 0x7f135c19bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:14:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 >> 172.21.15.33:0/3865242008 conn(0x7f135c076d10 msgr2=0x7f135c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:14:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 shutdown_connections 2024-01-25T16:14:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:14:48.254+0000 7f1364236640 1 -- 172.21.15.33:0/3865242008 wait complete. 2024-01-25T16:14:48.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 316274 6.17 MB/sec execute 225 sec latency 36.730 ms 2024-01-25T16:14:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:49 smithi033 ceph-mon[31417]: pgmap v215: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:49.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:49 smithi152 ceph-mon[36015]: pgmap v215: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:49 smithi076 ceph-mon[35642]: pgmap v215: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:49.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 318676 6.20 MB/sec execute 226 sec latency 20.886 ms 2024-01-25T16:14:50.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 320681 6.23 MB/sec execute 227 sec latency 28.893 ms 2024-01-25T16:14:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:51 smithi033 ceph-mon[31417]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:51.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:51 smithi152 ceph-mon[36015]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:51 smithi076 ceph-mon[35642]: pgmap v216: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 213 op/s 2024-01-25T16:14:51.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 322932 6.25 MB/sec execute 228 sec latency 82.923 ms 2024-01-25T16:14:52.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 324710 6.26 MB/sec execute 229 sec latency 16.774 ms 2024-01-25T16:14:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:53 smithi033 ceph-mon[31417]: pgmap v217: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 405 op/s 2024-01-25T16:14:53.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:53 smithi152 ceph-mon[36015]: pgmap v217: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 405 op/s 2024-01-25T16:14:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:53 smithi076 ceph-mon[35642]: pgmap v217: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.5 MiB/s wr, 405 op/s 2024-01-25T16:14:53.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 326755 6.29 MB/sec execute 230 sec latency 67.230 ms 2024-01-25T16:14:54.777 INFO:tasks.workunit.client.0.smithi033.stdout: 1 328829 6.29 MB/sec execute 231 sec latency 17.775 ms 2024-01-25T16:14:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:55 smithi033 ceph-mon[31417]: pgmap v218: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-01-25T16:14:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:55 smithi152 ceph-mon[36015]: pgmap v218: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-01-25T16:14:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:55 smithi076 ceph-mon[35642]: pgmap v218: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-01-25T16:14:55.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 330835 6.32 MB/sec execute 232 sec latency 18.617 ms 2024-01-25T16:14:56.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 333547 6.37 MB/sec execute 233 sec latency 14.008 ms 2024-01-25T16:14:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:57 smithi033 ceph-mon[31417]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-01-25T16:14:57.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:57 smithi152 ceph-mon[36015]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-01-25T16:14:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:57 smithi076 ceph-mon[35642]: pgmap v219: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.4 MiB/s wr, 404 op/s 2024-01-25T16:14:57.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 335120 6.37 MB/sec execute 234 sec latency 51.616 ms 2024-01-25T16:14:58.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 337243 6.40 MB/sec execute 235 sec latency 87.970 ms 2024-01-25T16:14:59.470 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:14:59 smithi076 ceph-mon[35642]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:14:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:14:59 smithi033 ceph-mon[31417]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:14:59.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:14:59 smithi152 ceph-mon[36015]: pgmap v220: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:14:59.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 339355 6.40 MB/sec execute 236 sec latency 19.178 ms 2024-01-25T16:15:00.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 340754 6.42 MB/sec execute 237 sec latency 26.513 ms 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3953653635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58104b90 msgr2=0x7fcd58104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3953653635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58104b90 0x7fcd58104f70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009900 tx=0x7fcd4c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3953653635 shutdown_connections 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3953653635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd58100800 0x7fcd5810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3953653635 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd580ffe40 0x7fcd581002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3953653635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58104b90 0x7fcd58104f70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3953653635 >> 172.21.15.33:0/3953653635 conn(0x7fcd58076a40 msgr2=0x7fcd58076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3953653635 shutdown_connections 2024-01-25T16:15:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.346+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3953653635 wait complete. 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 Processor -- start 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 -- start start 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 0x7fcd581980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd58104b90 0x7fcd581a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58198e20 con 0x7fcd58100800 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58198f90 con 0x7fcd580ffe40 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5cf0f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd581a3710 con 0x7fcd58104b90 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57776/0 (socket says 172.21.15.33:57776) 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 -- 172.21.15.33:0/3649257646 learned_addr learned my addr 172.21.15.33:0/3649257646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5659c640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 0x7fcd581980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd58104b90 0x7fcd581a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 -- 172.21.15.33:0/3649257646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd58104b90 msgr2=0x7fcd581a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd58104b90 0x7fcd581a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 -- 172.21.15.33:0/3649257646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 msgr2=0x7fcd581980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 0x7fcd581980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd55d9b640 1 -- 172.21.15.33:0/3649257646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4c009560 con 0x7fcd58100800 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.347+0000 7fcd5659c640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 0x7fcd581980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9f0 tx=0x7fcd4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd477fe640 1 -- 172.21.15.33:0/3649257646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48009700 con 0x7fcd58100800 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd581a39f0 con 0x7fcd58100800 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd477fe640 1 -- 172.21.15.33:0/3649257646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4800cd80 con 0x7fcd58100800 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd477fe640 1 -- 172.21.15.33:0/3649257646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48018860 con 0x7fcd58100800 2024-01-25T16:15:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.348+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd581a3f00 con 0x7fcd58100800 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.350+0000 7fcd477fe640 1 -- 172.21.15.33:0/3649257646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fcd480189c0 con 0x7fcd58100800 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.350+0000 7fcd5fbaa740 1 Processor -- start 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 -- start start 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc069b0 0x55666dc0f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd477fe640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 0x7fcd400a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc11b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666db75070 con 0x55666dc0f780 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666db72760 con 0x55666dc06090 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666db6e050 con 0x55666dc069b0 2024-01-25T16:15:01.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd5659c640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 0x7fcd400a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:44040/0 (socket says 172.21.15.33:44040) 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 -- 172.21.15.33:0/3102656590 learned_addr learned my addr 172.21.15.33:0/3102656590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd477fe640 1 -- 172.21.15.33:0/3649257646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcd48014070 con 0x7fcd58100800 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc11b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc069b0 0x55666dc0f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 -- 172.21.15.33:0/3102656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc069b0 msgr2=0x55666dc0f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc069b0 0x55666dc0f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 -- 172.21.15.33:0/3102656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 msgr2=0x55666dc11b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc11b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.351+0000 7fcd56d9d640 1 -- 172.21.15.33:0/3102656590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55666db65e70 con 0x55666dc06090 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.352+0000 7fcd5659c640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 0x7fcd400a39b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c005e40 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.352+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005c40 tx=0x7fcd50005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248688bee6ddab54 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.352+0000 7fcd44ff9640 1 -- 172.21.15.33:0/3102656590 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50004b20 con 0x55666dc06090 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.352+0000 7fcd44ff9640 1 -- 172.21.15.33:0/3102656590 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd50003e30 con 0x55666dc06090 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.352+0000 7fcd44ff9640 1 -- 172.21.15.33:0/3102656590 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd50004050 con 0x55666dc06090 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/3102656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 msgr2=0x55666dc06470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005c40 tx=0x7fcd50005bd0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/3102656590 shutdown_connections 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc11b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc069b0 0x55666dc0f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/3102656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc06470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/3102656590 >> 172.21.15.33:0/3102656590 conn(0x55666dc009b0 msgr2=0x55666dc00d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/3102656590 shutdown_connections 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/3102656590 wait complete. 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.353+0000 7fcd5fbaa740 1 -- start start 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 Processor -- start 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc0cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666db271e0 con 0x55666dc0f780 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666db26e80 con 0x55666dc069b0 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666daec0c0 con 0x55666dc06090 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:53052/0 (socket says 172.21.15.33:53052) 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 -- 172.21.15.33:0/2224437793 learned_addr learned my addr 172.21.15.33:0/2224437793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc0cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5fbaa740 10 client.0 _ll_register_callbacks cb 0x55666db11ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 -- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 msgr2=0x55666dc0cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc0cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 -- 172.21.15.33:0/2224437793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 msgr2=0x55666dc0d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc0d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.354+0000 7fcd5659c640 1 -- 172.21.15.33:0/2224437793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd50019040 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5659c640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c035000 tx=0x7fcd4c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a291da08a0be13e7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c005000 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcd4c047040 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55666db67040 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4c005160 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 10 client.25525 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55666db6c6d0 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55666db67040 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcd4c03f440 con 0x55666dc06090 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 10 client.25525 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 10 client.25525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 20 client.25525 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc1f3c0 0x55666dc217e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 4 client.25525 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.355+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55666db66d60 con 0x55666dc1f3c0 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.356+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc1f3c0 0x55666dc217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.356+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc1f3c0 0x55666dc217e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff3f7d0f458fb36 server_cookie=a53ade7de2e13f73 in_seq=0 out_seq=0 2024-01-25T16:15:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.356+0000 7fcd3dffb640 10 client.25525 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd3dffb640 1 -- 172.21.15.33:0/2224437793 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55666db66d60 con 0x55666dc1f3c0 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd3dffb640 10 client.25525 handle_command_reply: tid=0 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 client.25525 shutdown 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 msgr2=0x55666dc0c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c035000 tx=0x7fcd4c009260 comp rx=0 tx=0).stop 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 shutdown_connections 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc1f3c0 0x55666dc217e0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc0f780 0x55666dc0d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc0cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd3dffb640 0 client.25525 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/2224437793 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc06090 0x55666dc0c6b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 >> 172.21.15.33:0/2224437793 conn(0x55666dc009b0 msgr2=0x55666dc02700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 shutdown_connections 2024-01-25T16:15:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/2224437793 wait complete. 2024-01-25T16:15:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.358+0000 7fcd5fbaa740 20 client.25525 trim_cache size 0 max 16384 2024-01-25T16:15:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.371+0000 7fcd5fbaa740 1 Processor -- start 2024-01-25T16:15:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.371+0000 7fcd5fbaa740 1 -- start start 2024-01-25T16:15:01.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc0b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dbff8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dc017b0 con 0x55666dc069b0 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dbfc2e0 con 0x55666dc06090 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dc1f1c0 con 0x55666dc0f780 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57800/0 (socket says 172.21.15.33:57800) 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 -- 172.21.15.33:0/4280657273 learned_addr learned my addr 172.21.15.33:0/4280657273 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5659c640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc0b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dbff8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 -- 172.21.15.33:0/4280657273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 msgr2=0x55666dbff8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dbff8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 -- 172.21.15.33:0/4280657273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 msgr2=0x55666dc0b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc0b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 -- 172.21.15.33:0/4280657273 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55666db09060 con 0x55666dc069b0 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd5659c640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc0b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd48096630 tx=0x7fcd4801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb607187a3fc88c5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd3ffff640 1 -- 172.21.15.33:0/4280657273 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4809a130 con 0x55666dc069b0 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd3ffff640 1 -- 172.21.15.33:0/4280657273 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4809e1d0 con 0x55666dc069b0 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.372+0000 7fcd3ffff640 1 -- 172.21.15.33:0/4280657273 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48096840 con 0x55666dc069b0 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/4280657273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 msgr2=0x55666dc0cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd48096630 tx=0x7fcd4801f000 comp rx=0 tx=0).stop 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/4280657273 shutdown_connections 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dbff8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc069b0 0x55666dc0cf50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/4280657273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc06090 0x55666dc0b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/4280657273 >> 172.21.15.33:0/4280657273 conn(0x55666dc009b0 msgr2=0x55666dbff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/4280657273 shutdown_connections 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/4280657273 wait complete. 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.373+0000 7fcd5fbaa740 1 -- start start 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 Processor -- start 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dc15900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dc758f0 con 0x55666dc06090 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dc75a30 con 0x55666dc069b0 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55666dc75b70 con 0x55666dc0f780 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5fbaa740 10 client.0 _ll_register_callbacks cb 0x7fcd50009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57806/0 (socket says 172.21.15.33:57806) 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 -- 172.21.15.33:0/1683799545 learned_addr learned my addr 172.21.15.33:0/1683799545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd56d9d640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dc15900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 -- 172.21.15.33:0/1683799545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 msgr2=0x55666dc15900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dc15900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 -- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 msgr2=0x55666dc153c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc153c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 -- 172.21.15.33:0/1683799545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55666db6c6d0 con 0x55666dc06090 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd5659c640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c004a90 tx=0x7fcd4c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ffc6bc32a1cf5f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.374+0000 7fcd3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c005200 con 0x55666dc06090 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcd4c02de40 con 0x55666dc06090 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd50003e30 con 0x55666dc06090 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd4c005360 con 0x55666dc06090 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 10 client.15978 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:01.383 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fcd581a3f00 con 0x55666dc06090 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd4c06f080 con 0x55666dc06090 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcd4c060480 con 0x55666dc06090 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 10 client.15978 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 10 client.15978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 20 client.15978 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc77020 0x55666dc79440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 4 client.15978 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55666d86d600 con 0x55666dc77020 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.375+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc77020 0x55666dc79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.376+0000 7fcd55d9b640 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc77020 0x55666dc79440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7dd57b6e9e1d048 server_cookie=77d15ded4597724d in_seq=0 out_seq=0 2024-01-25T16:15:01.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.376+0000 7fcd3dffb640 10 client.15978 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd3dffb640 1 -- 172.21.15.33:0/1683799545 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55666d86d600 con 0x55666dc77020 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd3dffb640 10 client.15978 handle_command_reply: tid=0 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 client.15978 shutdown 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 msgr2=0x55666dc14e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c004a90 tx=0x7fcd4c004b40 comp rx=0 tx=0).stop 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 shutdown_connections 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55666dc77020 0x55666dc79440 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55666dc0f780 0x55666dc15900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55666dc069b0 0x55666dc153c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 --2- 172.21.15.33:0/1683799545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55666dc06090 0x55666dc14e80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 >> 172.21.15.33:0/1683799545 conn(0x55666dc009b0 msgr2=0x55666dc019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd3dffb640 0 client.15978 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 shutdown_connections 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 1 -- 172.21.15.33:0/1683799545 wait complete. 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.487+0000 7fcd5fbaa740 20 client.15978 trim_cache size 0 max 16384 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:15:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 msgr2=0x7fcd400a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 0x7fcd400a39b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c006010 tx=0x7fcd4c005e40 comp rx=0 tx=0).stop 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 msgr2=0x7fcd58198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800e9f0 tx=0x7fcd4800eeb0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.493 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 shutdown_connections 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcd400a14f0 0x7fcd400a39b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd58104b90 0x7fcd581a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd58100800 0x7fcd58198610 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 --2- 172.21.15.33:0/3649257646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd580ffe40 0x7fcd581980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 >> 172.21.15.33:0/3649257646 conn(0x7fcd58076a40 msgr2=0x7fcd580fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 shutdown_connections 2024-01-25T16:15:01.494 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:01.488+0000 7fcd5cf0f640 1 -- 172.21.15.33:0/3649257646 wait complete. 2024-01-25T16:15:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:01 smithi033 ceph-mon[31417]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:15:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:01 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:15:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:01 smithi152 ceph-mon[36015]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:15:01.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:01 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:15:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:01 smithi076 ceph-mon[35642]: pgmap v221: 97 pgs: 97 active+clean; 76 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.0 MiB/s wr, 297 op/s 2024-01-25T16:15:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:01 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:15:01.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 342344 6.42 MB/sec execute 238 sec latency 66.751 ms 2024-01-25T16:15:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:02 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:15:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:02 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:15:02.565 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:02 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:15:02.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:02 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:15:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:02 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:15:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:02 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:15:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:02 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:15:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:02 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:15:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:02 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:15:02.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 344284 6.45 MB/sec execute 239 sec latency 107.598 ms 2024-01-25T16:15:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:03 smithi033 ceph-mon[31417]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-01-25T16:15:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:03 smithi033 ceph-mon[31417]: osdmap e93: 12 total, 12 up, 12 in 2024-01-25T16:15:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:03 smithi152 ceph-mon[36015]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-01-25T16:15:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:03 smithi152 ceph-mon[36015]: osdmap e93: 12 total, 12 up, 12 in 2024-01-25T16:15:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:03 smithi076 ceph-mon[35642]: pgmap v222: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.6 MiB/s wr, 438 op/s 2024-01-25T16:15:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:03 smithi076 ceph-mon[35642]: osdmap e93: 12 total, 12 up, 12 in 2024-01-25T16:15:03.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 345707 6.45 MB/sec execute 240 sec latency 49.035 ms 2024-01-25T16:15:04.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 348008 6.48 MB/sec execute 241 sec latency 14.644 ms 2024-01-25T16:15:05.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:05 smithi033 ceph-mon[31417]: pgmap v224: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-01-25T16:15:05.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:05 smithi152 ceph-mon[36015]: pgmap v224: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-01-25T16:15:05.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:05 smithi076 ceph-mon[35642]: pgmap v224: 97 pgs: 97 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.7 MiB/s wr, 296 op/s 2024-01-25T16:15:05.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 350108 6.48 MB/sec execute 242 sec latency 21.764 ms 2024-01-25T16:15:06.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 352136 6.51 MB/sec execute 243 sec latency 20.396 ms 2024-01-25T16:15:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:07 smithi033 ceph-mon[31417]: pgmap v225: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:07.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:07 smithi152 ceph-mon[36015]: pgmap v225: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:07 smithi076 ceph-mon[35642]: pgmap v225: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:07.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 354666 6.55 MB/sec execute 244 sec latency 51.570 ms 2024-01-25T16:15:08.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 356085 6.55 MB/sec execute 245 sec latency 45.075 ms 2024-01-25T16:15:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:09 smithi033 ceph-mon[31417]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:09.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:09 smithi152 ceph-mon[36015]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:09 smithi076 ceph-mon[35642]: pgmap v226: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:09.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 358409 6.57 MB/sec execute 246 sec latency 18.205 ms 2024-01-25T16:15:10.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 359941 6.57 MB/sec execute 247 sec latency 124.439 ms 2024-01-25T16:15:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:11 smithi033 ceph-mon[31417]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:11 smithi033 ceph-mon[31417]: osdmap e94: 12 total, 12 up, 12 in 2024-01-25T16:15:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:11 smithi152 ceph-mon[36015]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:11.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:11 smithi152 ceph-mon[36015]: osdmap e94: 12 total, 12 up, 12 in 2024-01-25T16:15:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:11 smithi076 ceph-mon[35642]: pgmap v227: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 396 op/s 2024-01-25T16:15:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:11 smithi076 ceph-mon[35642]: osdmap e94: 12 total, 12 up, 12 in 2024-01-25T16:15:11.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 362234 6.60 MB/sec execute 248 sec latency 13.155 ms 2024-01-25T16:15:12.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 364374 6.61 MB/sec execute 249 sec latency 25.518 ms 2024-01-25T16:15:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:13 smithi033 ceph-mon[31417]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-01-25T16:15:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:13 smithi152 ceph-mon[36015]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-01-25T16:15:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:13 smithi076 ceph-mon[35642]: pgmap v229: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-01-25T16:15:13.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 366050 6.63 MB/sec execute 250 sec latency 50.340 ms 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 -- 172.21.15.33:0/2935296670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b401018c0 msgr2=0x7f3b40105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/2935296670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b401018c0 0x7f3b40105900 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009930 tx=0x7f3b2c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 -- 172.21.15.33:0/2935296670 shutdown_connections 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/2935296670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 0x7f3b4010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/2935296670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b401018c0 0x7f3b40105900 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/2935296670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b40100f10 0x7f3b401012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 -- 172.21.15.33:0/2935296670 >> 172.21.15.33:0/2935296670 conn(0x7f3b400fc7b0 msgr2=0x7f3b400febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 -- 172.21.15.33:0/2935296670 shutdown_connections 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.580+0000 7f3b46cb2640 1 -- 172.21.15.33:0/2935296670 wait complete. 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 Processor -- start 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 -- start start 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 0x7f3b40199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 0x7f3b4019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4019e680 con 0x7f3b40100f10 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4019e7f0 con 0x7f3b4019dc90 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b46cb2640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4019e960 con 0x7f3b40105fe0 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42548/0 (socket says 172.21.15.33:42548) 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1635737398 learned_addr learned my addr 172.21.15.33:0/1635737398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:14.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b4524f640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 0x7f3b4019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.581+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 0x7f3b40199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1635737398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 msgr2=0x7f3b40199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 0x7f3b40199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1635737398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 msgr2=0x7f3b4019e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 0x7f3b4019e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1635737398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34009490 con 0x7f3b40100f10 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b4524f640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 0x7f3b4019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400ea60 tx=0x7f3b3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b3dffb640 1 -- 172.21.15.33:0/1635737398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3400cde0 con 0x7f3b40100f10 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c009560 con 0x7f3b40100f10 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b3dffb640 1 -- 172.21.15.33:0/1635737398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b340035f0 con 0x7f3b40100f10 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b3dffb640 1 -- 172.21.15.33:0/1635737398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340044e0 con 0x7f3b40100f10 2024-01-25T16:15:14.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.582+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b401a6220 con 0x7f3b40100f10 2024-01-25T16:15:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.584+0000 7f3b3dffb640 1 -- 172.21.15.33:0/1635737398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3b34003a60 con 0x7f3b40100f10 2024-01-25T16:15:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b3dffb640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 0x7f3b300a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b3dffb640 1 -- 172.21.15.33:0/1635737398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3b34014070 con 0x7f3b40100f10 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 0x7f3b300a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 Processor -- start 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 -- start start 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f644070 con 0x55816f6d5090 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f641760 con 0x55816f6d59b0 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b49874740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f63d050 con 0x55816f6de780 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42560/0 (socket says 172.21.15.33:42560) 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b44a4e640 1 -- 172.21.15.33:0/189660532 learned_addr learned my addr 172.21.15.33:0/189660532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:14.595 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b4524f640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 0x7f3b300a39b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c005fd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.585+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 -- 172.21.15.33:0/189660532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 msgr2=0x55816f6e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 -- 172.21.15.33:0/189660532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 msgr2=0x55816f6de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6de240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 -- 172.21.15.33:0/189660532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816f634e70 con 0x55816f6d5090 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3b34099e90 tx=0x7f3b3409a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4861c56974495dd7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b2affd640 1 -- 172.21.15.33:0/189660532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3409e2e0 con 0x55816f6d5090 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b2affd640 1 -- 172.21.15.33:0/189660532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b340a2520 con 0x55816f6d5090 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.586+0000 7f3b2affd640 1 -- 172.21.15.33:0/189660532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a2090 con 0x55816f6d5090 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- 172.21.15.33:0/189660532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 msgr2=0x55816f6d5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f3b34099e90 tx=0x7f3b3409a890 comp rx=0 tx=0).stop 2024-01-25T16:15:14.596 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- 172.21.15.33:0/189660532 shutdown_connections 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- 172.21.15.33:0/189660532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6d5470 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- 172.21.15.33:0/189660532 >> 172.21.15.33:0/189660532 conn(0x55816f6cf9b0 msgr2=0x55816f6cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- 172.21.15.33:0/189660532 shutdown_connections 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- 172.21.15.33:0/189660532 wait complete. 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- start start 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 Processor -- start 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6db710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6de780 0x55816f6dc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.597 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f5f61e0 con 0x55816f6d59b0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f5f5e80 con 0x55816f6de780 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.587+0000 7f3b49874740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f5bb0c0 con 0x55816f6d5090 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b49874740 10 client.0 _ll_register_callbacks cb 0x55816f5e0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42576/0 (socket says 172.21.15.33:42576) 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 -- 172.21.15.33:0/756107406 learned_addr learned my addr 172.21.15.33:0/756107406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b4524f640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6de780 0x55816f6dc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 -- 172.21.15.33:0/756107406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 msgr2=0x55816f6db710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6db710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 -- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6de780 msgr2=0x55816f6dc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6de780 0x55816f6dc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 -- 172.21.15.33:0/756107406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b401a6220 con 0x55816f6d59b0 2024-01-25T16:15:14.598 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3b4006a3a0 tx=0x7f3b2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e203650291af3431 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c004250 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3b2c047070 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55816f636040 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b2c0043b0 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b49874740 10 client.15990 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.588+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55816f63b6d0 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55816f636040 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b2c003950 con 0x55816f6d59b0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 10 client.15990 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 20 client.15990 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f6f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 4 client.15990 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55816f635d60 con 0x55816f6ee4a0 2024-01-25T16:15:14.599 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f6f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b1ffff640 10 client.15990 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.589+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f6f08c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80de0a745f8ab142 server_cookie=7caed602020eaab7 in_seq=0 out_seq=0 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b1ffff640 1 -- 172.21.15.33:0/756107406 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55816f635d60 con 0x55816f6ee4a0 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b1ffff640 10 client.15990 handle_command_reply: tid=0 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 client.15990 shutdown 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 msgr2=0x55816f6dbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3b4006a3a0 tx=0x7f3b2c009260 comp rx=0 tx=0).stop 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 shutdown_connections 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f6f08c0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6de780 0x55816f6dc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b1ffff640 0 client.15990 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d59b0 0x55816f6dbc50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 --2- 172.21.15.33:0/756107406 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6db710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.600 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 >> 172.21.15.33:0/756107406 conn(0x55816f6cf9b0 msgr2=0x55816f6d1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 shutdown_connections 2024-01-25T16:15:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.590+0000 7f3b49874740 1 -- 172.21.15.33:0/756107406 wait complete. 2024-01-25T16:15:14.601 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.591+0000 7f3b49874740 20 client.15990 trim_cache size 0 max 16384 2024-01-25T16:15:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.604+0000 7f3b49874740 1 Processor -- start 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 -- start start 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f636040 con 0x55816f6de780 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f6d4370 con 0x55816f6d59b0 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b49874740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f6cf870 con 0x55816f6d5090 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42584/0 (socket says 172.21.15.33:42584) 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 -- 172.21.15.33:0/3284103866 learned_addr learned my addr 172.21.15.33:0/3284103866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 -- 172.21.15.33:0/3284103866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 msgr2=0x55816f6dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.612 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 -- 172.21.15.33:0/3284103866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 msgr2=0x55816f6d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 -- 172.21.15.33:0/3284103866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55816f5d8060 con 0x55816f6de780 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b4524f640 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3b38012930 tx=0x7f3b38009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad718c703db2a779 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.605+0000 7f3b2a7fc640 1 -- 172.21.15.33:0/3284103866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38017070 con 0x55816f6de780 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b2a7fc640 1 -- 172.21.15.33:0/3284103866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b38018040 con 0x55816f6de780 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b2a7fc640 1 -- 172.21.15.33:0/3284103866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004290 con 0x55816f6de780 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 -- 172.21.15.33:0/3284103866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 msgr2=0x55816f6da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3b38012930 tx=0x7f3b38009500 comp rx=0 tx=0).stop 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 -- 172.21.15.33:0/3284103866 shutdown_connections 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6de780 0x55816f6da520 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 --2- 172.21.15.33:0/3284103866 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6d5090 0x55816f6dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 -- 172.21.15.33:0/3284103866 >> 172.21.15.33:0/3284103866 conn(0x55816f6cf9b0 msgr2=0x55816f6f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 -- 172.21.15.33:0/3284103866 shutdown_connections 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.606+0000 7f3b49874740 1 -- 172.21.15.33:0/3284103866 wait complete. 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 -- start start 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 Processor -- start 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6e4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f6e5dc0 con 0x55816f6d5090 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f7449e0 con 0x55816f6d59b0 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55816f744ba0 con 0x55816f6de780 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b49874740 10 client.0 _ll_register_callbacks cb 0x7f3b34095e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42600/0 (socket says 172.21.15.33:42600) 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1125428034 learned_addr learned my addr 172.21.15.33:0/1125428034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b4524f640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6e4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1125428034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 msgr2=0x55816f6e4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 msgr2=0x55816f6e4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6e4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.607+0000 7f3b44a4e640 1 -- 172.21.15.33:0/1125428034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b380125e0 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b44a4e640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b3401f000 tx=0x7f3b34003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0131eb40e45d50 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34003760 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3b340b7070 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b340a2520 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b340038c0 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 10 client.15996 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55816f5fda80 con 0x55816f6d5090 2024-01-25T16:15:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b340a6020 con 0x55816f6d5090 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3b34095f80 con 0x55816f6d5090 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 10 client.15996 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 20 client.15996 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f7480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 4 client.15996 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55816f33c600 con 0x55816f6ee4a0 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.608+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f7480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.609+0000 7f3b1ffff640 10 client.15996 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.609+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f7480b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368f96de3fb63315 server_cookie=fd2f2ec554966b6e in_seq=0 out_seq=0 2024-01-25T16:15:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b1ffff640 1 -- 172.21.15.33:0/1125428034 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55816f33c600 con 0x55816f6ee4a0 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b1ffff640 10 client.15996 handle_command_reply: tid=0 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 client.15996 shutdown 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 msgr2=0x55816f6e3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3b3401f000 tx=0x7f3b34003d60 comp rx=0 tx=0).stop 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 shutdown_connections 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55816f6ee4a0 0x55816f7480b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55816f6de780 0x55816f6e4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55816f6d59b0 0x55816f6e4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 --2- 172.21.15.33:0/1125428034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55816f6d5090 0x55816f6e3ec0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 >> 172.21.15.33:0/1125428034 conn(0x55816f6cf9b0 msgr2=0x55816f6d9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.713+0000 7f3b1ffff640 0 client.15996 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 shutdown_connections 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b49874740 1 -- 172.21.15.33:0/1125428034 wait complete. 2024-01-25T16:15:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b49874740 20 client.15996 trim_cache size 0 max 16384 2024-01-25T16:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:15:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 msgr2=0x7f3b300a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 0x7f3b300a39b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c000c00 tx=0x7f3b2c005fd0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 msgr2=0x7f3b40199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.714+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400ea60 tx=0x7f3b3400ef20 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 shutdown_connections 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3b300a14f0 0x7f3b300a39b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b4019dc90 0x7f3b4019e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3b40105fe0 0x7f3b40199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 --2- 172.21.15.33:0/1635737398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b40100f10 0x7f3b40199420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 >> 172.21.15.33:0/1635737398 conn(0x7f3b400fc7b0 msgr2=0x7f3b400fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 shutdown_connections 2024-01-25T16:15:14.719 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:14.715+0000 7f3b46cb2640 1 -- 172.21.15.33:0/1635737398 wait complete. 2024-01-25T16:15:14.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 368254 6.65 MB/sec execute 251 sec latency 100.280 ms 2024-01-25T16:15:15.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:15 smithi033 ceph-mon[31417]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-01-25T16:15:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:15 smithi152 ceph-mon[36015]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-01-25T16:15:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:15 smithi076 ceph-mon[35642]: pgmap v230: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.7 MiB/s wr, 349 op/s 2024-01-25T16:15:15.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 370268 6.67 MB/sec execute 252 sec latency 13.741 ms 2024-01-25T16:15:16.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 372652 6.70 MB/sec execute 253 sec latency 14.190 ms 2024-01-25T16:15:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:17 smithi033 ceph-mon[31417]: pgmap v231: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:17 smithi152 ceph-mon[36015]: pgmap v231: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:17 smithi076 ceph-mon[35642]: pgmap v231: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:17.778 INFO:tasks.workunit.client.0.smithi033.stdout: 1 374329 6.69 MB/sec execute 254 sec latency 54.472 ms 2024-01-25T16:15:18.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 376370 6.72 MB/sec execute 255 sec latency 83.219 ms 2024-01-25T16:15:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:19 smithi033 ceph-mon[31417]: pgmap v232: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:19.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:19 smithi152 ceph-mon[36015]: pgmap v232: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:19 smithi076 ceph-mon[35642]: pgmap v232: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:19.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 378310 6.72 MB/sec execute 256 sec latency 53.855 ms 2024-01-25T16:15:20.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 380251 6.74 MB/sec execute 257 sec latency 17.219 ms 2024-01-25T16:15:21.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:21 smithi033 ceph-mon[31417]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:21 smithi152 ceph-mon[36015]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:21 smithi076 ceph-mon[35642]: pgmap v233: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-01-25T16:15:21.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 382749 6.77 MB/sec execute 258 sec latency 14.550 ms 2024-01-25T16:15:22.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 384627 6.79 MB/sec execute 259 sec latency 16.733 ms 2024-01-25T16:15:23.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:23 smithi033 ceph-mon[31417]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-01-25T16:15:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:23 smithi152 ceph-mon[36015]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-01-25T16:15:23.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:23 smithi076 ceph-mon[35642]: pgmap v234: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-01-25T16:15:23.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 386925 6.81 MB/sec execute 260 sec latency 87.485 ms 2024-01-25T16:15:24.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 388368 6.81 MB/sec execute 261 sec latency 36.200 ms 2024-01-25T16:15:25.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:25 smithi033 ceph-mon[31417]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-01-25T16:15:25.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:25 smithi152 ceph-mon[36015]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-01-25T16:15:25.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:25 smithi076 ceph-mon[35642]: pgmap v235: 97 pgs: 97 active+clean; 81 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 289 op/s 2024-01-25T16:15:25.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 390317 6.83 MB/sec execute 262 sec latency 13.228 ms 2024-01-25T16:15:26.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 391996 6.83 MB/sec execute 263 sec latency 17.700 ms 2024-01-25T16:15:27.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:27 smithi033 ceph-mon[31417]: pgmap v236: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 388 op/s 2024-01-25T16:15:27.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:27 smithi152 ceph-mon[36015]: pgmap v236: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 388 op/s 2024-01-25T16:15:27.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:27 smithi076 ceph-mon[35642]: pgmap v236: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 388 op/s 2024-01-25T16:15:27.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 394221 6.85 MB/sec execute 264 sec latency 14.677 ms 2024-01-25T16:15:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 -- 172.21.15.33:0/1862105143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc0ffe40 msgr2=0x7f5dbc1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 --2- 172.21.15.33:0/1862105143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1002c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009900 tx=0x7f5db002bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 -- 172.21.15.33:0/1862105143 shutdown_connections 2024-01-25T16:15:27.811 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 --2- 172.21.15.33:0/1862105143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc100800 0x7f5dbc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 --2- 172.21.15.33:0/1862105143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1002c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 --2- 172.21.15.33:0/1862105143 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 0x7f5dbc104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 -- 172.21.15.33:0/1862105143 >> 172.21.15.33:0/1862105143 conn(0x7f5dbc076a40 msgr2=0x7f5dbc076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 -- 172.21.15.33:0/1862105143 shutdown_connections 2024-01-25T16:15:27.812 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.808+0000 7f5dc3451640 1 -- 172.21.15.33:0/1862105143 wait complete. 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 Processor -- start 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 -- start start 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 0x7f5dbc19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc19e2a0 con 0x7f5dbc100800 2024-01-25T16:15:27.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc19e410 con 0x7f5dbc0ffe40 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc3451640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dbc19e5d0 con 0x7f5dbc104b90 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44716/0 (socket says 172.21.15.33:44716) 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.809+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2204032870 learned_addr learned my addr 172.21.15.33:0/2204032870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 0x7f5dbc19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2204032870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 msgr2=0x7f5dbc19db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 0x7f5dbc19db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2204032870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 msgr2=0x7f5dbc1a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2204032870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009560 con 0x7f5dbc100800 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005940 tx=0x7f5db002c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dba7fc640 1 -- 172.21.15.33:0/2204032870 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004250 con 0x7f5dbc100800 2024-01-25T16:15:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dba7fc640 1 -- 172.21.15.33:0/2204032870 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db00043b0 con 0x7f5dbc100800 2024-01-25T16:15:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dbc19e850 con 0x7f5dbc100800 2024-01-25T16:15:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dba7fc640 1 -- 172.21.15.33:0/2204032870 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003b90 con 0x7f5dbc100800 2024-01-25T16:15:27.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.810+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dbc1aa500 con 0x7f5dbc100800 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.812+0000 7f5dba7fc640 1 -- 172.21.15.33:0/2204032870 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5db0003cf0 con 0x7f5dbc100800 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 Processor -- start 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 -- start start 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dba7fc640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 0x7f5da40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e401f070 con 0x55b8e40b0090 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e401c760 con 0x55b8e40b09b0 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc600e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e4018050 con 0x55b8e40b9780 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dba7fc640 1 -- 172.21.15.33:0/2204032870 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5db002de40 con 0x7f5dbc100800 2024-01-25T16:15:27.823 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 0x7f5da40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44722/0 (socket says 172.21.15.33:44722) 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.813+0000 7f5dc11ed640 1 -- 172.21.15.33:0/2858555460 learned_addr learned my addr 172.21.15.33:0/2858555460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 -- 172.21.15.33:0/2858555460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 msgr2=0x55b8e40bbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 -- 172.21.15.33:0/2858555460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 msgr2=0x55b8e40b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 -- 172.21.15.33:0/2858555460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e400fe70 con 0x55b8e40b0090 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5db40114c0 tx=0x7f5db4011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff4352353ca41d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 0x7f5da40a39b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5dac005fd0 tx=0x7f5dac009f00 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.824 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5da37fe640 1 -- 172.21.15.33:0/2858555460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4004020 con 0x55b8e40b0090 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5da37fe640 1 -- 172.21.15.33:0/2858555460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db40035f0 con 0x55b8e40b0090 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.814+0000 7f5da37fe640 1 -- 172.21.15.33:0/2858555460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400f840 con 0x55b8e40b0090 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- 172.21.15.33:0/2858555460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 msgr2=0x55b8e40b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f5db40114c0 tx=0x7f5db4011980 comp rx=0 tx=0).stop 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- 172.21.15.33:0/2858555460 shutdown_connections 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2858555460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b0470 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- 172.21.15.33:0/2858555460 >> 172.21.15.33:0/2858555460 conn(0x55b8e40aa9b0 msgr2=0x55b8e40aad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- 172.21.15.33:0/2858555460 shutdown_connections 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- 172.21.15.33:0/2858555460 wait complete. 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 -- start start 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.815+0000 7f5dc600e740 1 Processor -- start 2024-01-25T16:15:27.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e3fd11e0 con 0x55b8e40b09b0 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e3fd0e80 con 0x55b8e40b0090 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e3f960c0 con 0x55b8e40b9780 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc600e740 10 client.0 _ll_register_callbacks cb 0x55b8e3fbbac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:40410/0 (socket says 172.21.15.33:40410) 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44728/0 (socket says 172.21.15.33:44728) 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 -- 172.21.15.33:0/640825345 learned_addr learned my addr 172.21.15.33:0/640825345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 -- 172.21.15.33:0/640825345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 msgr2=0x55b8e40b71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 -- 172.21.15.33:0/640825345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 msgr2=0x55b8e40b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 -- 172.21.15.33:0/640825345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4009cc0 con 0x55b8e40b0090 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5db4011e80 tx=0x7f5db4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb734c168f4745a7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.816+0000 7f5da0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db4005230 con 0x55b8e40b0090 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8e4011040 con 0x55b8e40b0090 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5db4005390 con 0x55b8e40b0090 2024-01-25T16:15:27.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db4005550 con 0x55b8e40b0090 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 10 client.25526 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b8e40166d0 con 0x55b8e40b0090 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8e4011040 con 0x55b8e40b0090 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5db4004cc0 con 0x55b8e40b0090 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 10 client.25526 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 10 client.25526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 20 client.25526 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e40c9520 0x55b8e40cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 4 client.25526 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8e4010d60 con 0x55b8e40c9520 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.817+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e40c9520 0x55b8e40cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5da0ff9640 10 client.25526 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e40c9520 0x55b8e40cb930 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5edc8087c1842737 server_cookie=342747fc2d317ba3 in_seq=0 out_seq=0 2024-01-25T16:15:27.828 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5da0ff9640 1 -- 172.21.15.33:0/640825345 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8e4010d60 con 0x55b8e40c9520 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5da0ff9640 10 client.25526 handle_command_reply: tid=0 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5dc600e740 1 client.25526 shutdown 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 msgr2=0x55b8e40b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.818+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5db4011e80 tx=0x7f5db4005040 comp rx=0 tx=0).stop 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 shutdown_connections 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e40c9520 0x55b8e40cb930 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5da0ff9640 0 client.25526 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 --2- 172.21.15.33:0/640825345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40b6760 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 >> 172.21.15.33:0/640825345 conn(0x55b8e40aa9b0 msgr2=0x55b8e40ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 shutdown_connections 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 1 -- 172.21.15.33:0/640825345 wait complete. 2024-01-25T16:15:27.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.819+0000 7f5dc600e740 20 client.25526 trim_cache size 0 max 16384 2024-01-25T16:15:27.842 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 Processor -- start 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 -- start start 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40a60e0 con 0x55b8e40b0090 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40a62e0 con 0x55b8e40b09b0 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc600e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40aa820 con 0x55b8e40b9780 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc19ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:40420/0 (socket says 172.21.15.33:40420) 2024-01-25T16:15:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc11ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44736/0 (socket says 172.21.15.33:44736) 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2078355688 learned_addr learned my addr 172.21.15.33:0/2078355688 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2078355688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 msgr2=0x55b8e40b71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2078355688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 msgr2=0x55b8e40b8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc09ec640 1 -- 172.21.15.33:0/2078355688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8e3fb3060 con 0x55b8e40b09b0 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.833+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5db00b8af0 tx=0x7f5db002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11bb40677215e842 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5da2ffd640 1 -- 172.21.15.33:0/2078355688 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00d8070 con 0x55b8e40b09b0 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5da2ffd640 1 -- 172.21.15.33:0/2078355688 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db00473b0 con 0x55b8e40b09b0 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5da2ffd640 1 -- 172.21.15.33:0/2078355688 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00ba730 con 0x55b8e40b09b0 2024-01-25T16:15:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 -- 172.21.15.33:0/2078355688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 msgr2=0x55b8e40b6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5db00b8af0 tx=0x7f5db002bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 -- 172.21.15.33:0/2078355688 shutdown_connections 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40b71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b09b0 0x55b8e40b6cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 --2- 172.21.15.33:0/2078355688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b0090 0x55b8e40b8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.834+0000 7f5dc600e740 1 -- 172.21.15.33:0/2078355688 >> 172.21.15.33:0/2078355688 conn(0x55b8e40aa9b0 msgr2=0x55b8e40b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- 172.21.15.33:0/2078355688 shutdown_connections 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- 172.21.15.33:0/2078355688 wait complete. 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- start start 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 Processor -- start 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40c0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40bf130 con 0x55b8e40b09b0 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40bf2a0 con 0x55b8e40b0090 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8e40bf460 con 0x55b8e40b9780 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc600e740 10 client.0 _ll_register_callbacks cb 0x7f5db4009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc09ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44752/0 (socket says 172.21.15.33:44752) 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc09ec640 1 -- 172.21.15.33:0/3674487831 learned_addr learned my addr 172.21.15.33:0/3674487831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40c0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.835+0000 7f5dc19ee640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 -- 172.21.15.33:0/3674487831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 msgr2=0x55b8e40bea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 -- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 msgr2=0x55b8e40c0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40c0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 -- 172.21.15.33:0/3674487831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8e40166d0 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc09ec640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005e90 tx=0x7f5db00ece40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f18d92eb481a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00d8050 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5db40035f0 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5db00bccf0 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5db00bc5f0 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc600e740 10 client.16014 fetch_fsmap learned FSMap version 26 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f5dbc1aa500 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5db40035f0 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 26) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5db003c070 con 0x55b8e40b09b0 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc600e740 10 client.16014 fetch_fsmap finished waiting for FSMap version 26 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.836+0000 7f5dc600e740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc600e740 20 client.16014 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e4120a70 0x55b8e4122e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc600e740 4 client.16014 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:15:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8e3d17600 con 0x55b8e4120a70 2024-01-25T16:15:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e4120a70 0x55b8e4122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5dc11ed640 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e4120a70 0x55b8e4122e90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25984447c8f8d5d server_cookie=16ed4e088f874120 in_seq=0 out_seq=0 2024-01-25T16:15:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.837+0000 7f5da0ff9640 10 client.16014 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5da0ff9640 1 -- 172.21.15.33:0/3674487831 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8e3d17600 con 0x55b8e4120a70 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5da0ff9640 10 client.16014 handle_command_reply: tid=0 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 client.16014 shutdown 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 msgr2=0x55b8e40c0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005e90 tx=0x7f5db00ece40 comp rx=0 tx=0).stop 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 shutdown_connections 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55b8e4120a70 0x55b8e4122e90 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55b8e40b9780 0x55b8e40bea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b8e40b09b0 0x55b8e40c0980 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5da0ff9640 0 client.16014 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:27.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 --2- 172.21.15.33:0/3674487831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b8e40b0090 0x55b8e40c0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.977+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 >> 172.21.15.33:0/3674487831 conn(0x55b8e40aa9b0 msgr2=0x55b8e40a9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 shutdown_connections 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc600e740 1 -- 172.21.15.33:0/3674487831 wait complete. 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc600e740 20 client.16014 trim_cache size 0 max 16384 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:15:27.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 msgr2=0x7f5da40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 0x7f5da40a39b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5dac005fd0 tx=0x7f5dac009f00 comp rx=0 tx=0).stop 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 msgr2=0x7f5dbc1a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5db0005940 tx=0x7f5db002c670 comp rx=0 tx=0).stop 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 shutdown_connections 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5da40a14f0 0x7f5da40a39b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5dbc104b90 0x7f5dbc19db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dbc100800 0x7f5dbc1a3ac0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 --2- 172.21.15.33:0/2204032870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dbc0ffe40 0x7f5dbc1a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.978+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 >> 172.21.15.33:0/2204032870 conn(0x7f5dbc076a40 msgr2=0x7f5dbc068470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.979+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 shutdown_connections 2024-01-25T16:15:27.984 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:27.979+0000 7f5dc3451640 1 -- 172.21.15.33:0/2204032870 wait complete. 2024-01-25T16:15:28.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 396059 6.85 MB/sec execute 265 sec latency 101.603 ms 2024-01-25T16:15:28.882 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:15:28 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:15:28.883 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:15:28 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:15:28.883 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:15:28 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/924430971 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5589b8eb5520 con 0x5589b8e8b000 2024-01-25T16:15:28.883 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:15:28 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:15:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:29 smithi033 ceph-mon[31417]: pgmap v237: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-01-25T16:15:29.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:29 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:15:29.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:29 smithi152 ceph-mon[36015]: pgmap v237: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-01-25T16:15:29.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:29 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:15:29.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:29 smithi076 ceph-mon[35642]: pgmap v237: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 204 op/s 2024-01-25T16:15:29.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:29 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:15:29.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 397721 6.88 MB/sec execute 266 sec latency 113.251 ms 2024-01-25T16:15:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:30 smithi033 ceph-mon[31417]: osdmap e95: 12 total, 12 up, 12 in 2024-01-25T16:15:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:30 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/2958940148,v1:172.21.15.152:6837/2958940148] up:boot 2024-01-25T16:15:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:30 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:15:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:30 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:15:30.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:30 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:15:30.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:30 smithi152 ceph-mon[36015]: osdmap e95: 12 total, 12 up, 12 in 2024-01-25T16:15:30.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:30 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/2958940148,v1:172.21.15.152:6837/2958940148] up:boot 2024-01-25T16:15:30.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:30 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:15:30.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:30 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:15:30.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:30 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:15:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:30 smithi076 ceph-mon[35642]: osdmap e95: 12 total, 12 up, 12 in 2024-01-25T16:15:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:30 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/2958940148,v1:172.21.15.152:6837/2958940148] up:boot 2024-01-25T16:15:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:30 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:15:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:30 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:15:30.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:30 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:15:30.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 399734 6.88 MB/sec execute 267 sec latency 116.720 ms 2024-01-25T16:15:31.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:31 smithi033 ceph-mon[31417]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 245 op/s 2024-01-25T16:15:31.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:31 smithi152 ceph-mon[36015]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 245 op/s 2024-01-25T16:15:31.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:31 smithi076 ceph-mon[35642]: pgmap v239: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.6 MiB/s wr, 245 op/s 2024-01-25T16:15:31.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 401600 6.90 MB/sec execute 268 sec latency 39.502 ms 2024-01-25T16:15:32.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:32 smithi033 ceph-mon[31417]: pgmap v240: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:32.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:32 smithi152 ceph-mon[36015]: pgmap v240: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:32.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:32 smithi076 ceph-mon[35642]: pgmap v240: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:32.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 403855 6.92 MB/sec execute 269 sec latency 82.869 ms 2024-01-25T16:15:33.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 405835 6.93 MB/sec execute 270 sec latency 17.761 ms 2024-01-25T16:15:34.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:34 smithi033 ceph-mon[31417]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:34.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:34 smithi152 ceph-mon[36015]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:34.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:34 smithi076 ceph-mon[35642]: pgmap v241: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.6 MiB/s wr, 333 op/s 2024-01-25T16:15:34.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 408157 6.96 MB/sec execute 271 sec latency 90.790 ms 2024-01-25T16:15:35.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 410040 6.96 MB/sec execute 272 sec latency 22.072 ms 2024-01-25T16:15:36.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 412084 6.98 MB/sec execute 273 sec latency 12.579 ms 2024-01-25T16:15:37.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:37 smithi033 ceph-mon[31417]: pgmap v242: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:37.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:37 smithi152 ceph-mon[36015]: pgmap v242: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:37.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:37 smithi076 ceph-mon[35642]: pgmap v242: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:37.779 INFO:tasks.workunit.client.0.smithi033.stdout: 1 414472 7.00 MB/sec execute 274 sec latency 15.403 ms 2024-01-25T16:15:38.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 416493 7.02 MB/sec execute 275 sec latency 16.762 ms 2024-01-25T16:15:39.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:39 smithi033 ceph-mon[31417]: pgmap v243: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:39.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:39 smithi152 ceph-mon[36015]: pgmap v243: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:39.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:39 smithi076 ceph-mon[35642]: pgmap v243: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.2 MiB/s wr, 334 op/s 2024-01-25T16:15:39.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 418679 7.04 MB/sec execute 276 sec latency 57.772 ms 2024-01-25T16:15:40.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 420409 7.04 MB/sec execute 277 sec latency 18.632 ms 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 -- 172.21.15.33:0/2614624587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784103920 msgr2=0x7f9784107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 --2- 172.21.15.33:0/2614624587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784103920 0x7f9784107960 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9770009930 tx=0x7f977002bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 -- 172.21.15.33:0/2614624587 shutdown_connections 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 --2- 172.21.15.33:0/2614624587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 0x7f978410fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 --2- 172.21.15.33:0/2614624587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784103920 0x7f9784107960 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 --2- 172.21.15.33:0/2614624587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9784102f70 0x7f9784103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.075 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 -- 172.21.15.33:0/2614624587 >> 172.21.15.33:0/2614624587 conn(0x7f97840fe7d0 msgr2=0x7f9784100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 -- 172.21.15.33:0/2614624587 shutdown_connections 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.070+0000 7f9789b9e640 1 -- 172.21.15.33:0/2614624587 wait complete. 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 Processor -- start 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 -- start start 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 0x7f9784199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f978419db90 0x7f978419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e580 con 0x7f9784102f70 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e6f0 con 0x7f9784108040 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9789b9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978419e860 con 0x7f978419db90 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46932/0 (socket says 172.21.15.33:46932) 2024-01-25T16:15:41.076 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9788b9c640 1 -- 172.21.15.33:0/407921893 learned_addr learned my addr 172.21.15.33:0/407921893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f9783fff640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 0x7f9784199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.071+0000 7f978939d640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f978419db90 0x7f978419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 -- 172.21.15.33:0/407921893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f978419db90 msgr2=0x7f978419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f978419db90 0x7f978419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 -- 172.21.15.33:0/407921893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 msgr2=0x7f9784199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 0x7f9784199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 -- 172.21.15.33:0/407921893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770009560 con 0x7f9784102f70 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9788b9c640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f977800d770 tx=0x7f977800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9781ffb640 1 -- 172.21.15.33:0/407921893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778009c30 con 0x7f9784102f70 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9781ffb640 1 -- 172.21.15.33:0/407921893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f977800de80 con 0x7f9784102f70 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97841a5e40 con 0x7f9784102f70 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9781ffb640 1 -- 172.21.15.33:0/407921893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778004de0 con 0x7f9784102f70 2024-01-25T16:15:41.077 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.072+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97841a6210 con 0x7f9784102f70 2024-01-25T16:15:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.074+0000 7f9781ffb640 1 -- 172.21.15.33:0/407921893 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f9778004f40 con 0x7f9784102f70 2024-01-25T16:15:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 Processor -- start 2024-01-25T16:15:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9781ffb640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 0x7f97740a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 -- start start 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978939d640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 0x7f97740a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9781ffb640 1 -- 172.21.15.33:0/407921893 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f97780cc950 con 0x7f9784102f70 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d57410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d59d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5cbd070 con 0x5635d5d4e260 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5cba760 con 0x5635d5d4eb80 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f978d9c4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5cb6050 con 0x5635d5d57950 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46942/0 (socket says 172.21.15.33:46942) 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9788b9c640 1 -- 172.21.15.33:0/2158665634 learned_addr learned my addr 172.21.15.33:0/2158665634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.075+0000 7f9783fff640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d57410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f978939d640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d59d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 -- 172.21.15.33:0/2158665634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 msgr2=0x5635d5d59d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d59d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 -- 172.21.15.33:0/2158665634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 msgr2=0x5635d5d57410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d57410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 -- 172.21.15.33:0/2158665634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d5cade70 con 0x5635d5d4e260 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9783fff640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d57410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f978939d640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 0x7f97740a3900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f977c005fd0 tx=0x7f977c011810 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9778095cf0 tx=0x7f9778099940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d369ce9641f799 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f976effd640 1 -- 172.21.15.33:0/2158665634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778097b90 con 0x5635d5d4e260 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f976effd640 1 -- 172.21.15.33:0/2158665634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97780ca310 con 0x5635d5d4e260 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.076+0000 7f976effd640 1 -- 172.21.15.33:0/2158665634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780a1640 con 0x5635d5d4e260 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- 172.21.15.33:0/2158665634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 msgr2=0x5635d5d4e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9778095cf0 tx=0x7f9778099940 comp rx=0 tx=0).stop 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- 172.21.15.33:0/2158665634 shutdown_connections 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d59d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d57410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2158665634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d4e640 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- 172.21.15.33:0/2158665634 >> 172.21.15.33:0/2158665634 conn(0x5635d5d48b80 msgr2=0x5635d5d48f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- 172.21.15.33:0/2158665634 shutdown_connections 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- 172.21.15.33:0/2158665634 wait complete. 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 -- start start 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.077+0000 7f978d9c4740 1 Processor -- start 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 0x5635d5d54910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d55390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5c6f1e0 con 0x5635d5d4eb80 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5c6ee80 con 0x5635d5d4e260 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5c340c0 con 0x5635d5d57950 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978d9c4740 10 client.0 _ll_register_callbacks cb 0x5635d5c59ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:41.087 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46956/0 (socket says 172.21.15.33:46956) 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 0x5635d5d54910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 -- 172.21.15.33:0/3341132036 learned_addr learned my addr 172.21.15.33:0/3341132036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978939d640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d55390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 -- 172.21.15.33:0/3341132036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 msgr2=0x5635d5d55390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d55390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 -- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 msgr2=0x5635d5d54910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 0x5635d5d54910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 -- 172.21.15.33:0/3341132036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97841a6210 con 0x5635d5d4eb80 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 0x5635d5d54910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f978939d640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d55390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9783fff640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef9c43643d7c36 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.088 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9763fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770004dd0 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.078+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9770046070 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635d5caf040 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9770004040 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5635d5cb46d0 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635d5caf040 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f977003c040 con 0x5635d5d4eb80 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 10 client.16029 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 20 client.16029 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5d67850 0x5635d5d69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 4 client.16029 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:15:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635d5caed60 con 0x5635d5d67850 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.079+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5d67850 0x5635d5d69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5d67850 0x5635d5d69c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0b474d4ac2c88a server_cookie=d957125c619413aa in_seq=0 out_seq=0 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f9763fff640 10 client.16029 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f9763fff640 1 -- 172.21.15.33:0/3341132036 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5635d5caed60 con 0x5635d5d67850 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f9763fff640 10 client.16029 handle_command_reply: tid=0 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 client.16029 shutdown 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 msgr2=0x5635d5d54e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f9770009900 tx=0x7f977002c9b0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 shutdown_connections 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5d67850 0x5635d5d69c70 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d57950 0x5635d5d55390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4eb80 0x5635d5d54e50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f9763fff640 0 client.16029 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3341132036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4e260 0x5635d5d54910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 >> 172.21.15.33:0/3341132036 conn(0x5635d5d48b80 msgr2=0x5635d5d4a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.080+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 shutdown_connections 2024-01-25T16:15:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.081+0000 7f978d9c4740 1 -- 172.21.15.33:0/3341132036 wait complete. 2024-01-25T16:15:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.081+0000 7f978d9c4740 20 client.16029 trim_cache size 0 max 16384 2024-01-25T16:15:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 Processor -- start 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 -- start start 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4eb80 0x5635d5d4a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d57950 0x5635d5d5e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5d43b30 con 0x5635d5d4e260 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5d4daa0 con 0x5635d5d57950 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978d9c4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5caf040 con 0x5635d5d4eb80 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46960/0 (socket says 172.21.15.33:46960) 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 -- 172.21.15.33:0/2696830304 learned_addr learned my addr 172.21.15.33:0/2696830304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f978939d640 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d57950 0x5635d5d5e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9783fff640 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4eb80 0x5635d5d4a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 -- 172.21.15.33:0/2696830304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4eb80 msgr2=0x5635d5d4a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4eb80 0x5635d5d4a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.095+0000 7f9788b9c640 1 -- 172.21.15.33:0/2696830304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d57950 msgr2=0x5635d5d5e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d57950 0x5635d5d5e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f9788b9c640 1 -- 172.21.15.33:0/2696830304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635d5c51060 con 0x5635d5d4e260 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f9788b9c640 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f97780ca140 tx=0x7f9778084040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6ae6f0e116e219 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f976e7fc640 1 -- 172.21.15.33:0/2696830304 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778097b90 con 0x5635d5d4e260 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f976e7fc640 1 -- 172.21.15.33:0/2696830304 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9778095f00 con 0x5635d5d4e260 2024-01-25T16:15:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.096+0000 7f976e7fc640 1 -- 172.21.15.33:0/2696830304 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780a18a0 con 0x5635d5d4e260 2024-01-25T16:15:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- 172.21.15.33:0/2696830304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 msgr2=0x5635d5d551b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f97780ca140 tx=0x7f9778084040 comp rx=0 tx=0).stop 2024-01-25T16:15:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- 172.21.15.33:0/2696830304 shutdown_connections 2024-01-25T16:15:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d57950 0x5635d5d5e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.103 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4eb80 0x5635d5d4a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- 172.21.15.33:0/2696830304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d4e260 0x5635d5d551b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- 172.21.15.33:0/2696830304 >> 172.21.15.33:0/2696830304 conn(0x5635d5d48b80 msgr2=0x5635d5d49fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- 172.21.15.33:0/2696830304 shutdown_connections 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- 172.21.15.33:0/2696830304 wait complete. 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- start start 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 Processor -- start 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4e260 0x5635d5d5d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5d5e320 con 0x5635d5d57950 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5dbe2a0 con 0x5635d5d4eb80 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635d5dbe3e0 con 0x5635d5d4e260 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.097+0000 7f978d9c4740 10 client.0 _ll_register_callbacks cb 0x7f9778099620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:41.104 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978939d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978939d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46976/0 (socket says 172.21.15.33:46976) 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978939d640 1 -- 172.21.15.33:0/3234415537 learned_addr learned my addr 172.21.15.33:0/3234415537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4e260 0x5635d5d5d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 -- 172.21.15.33:0/3234415537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4e260 msgr2=0x5635d5d5d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4e260 0x5635d5d5d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 -- 172.21.15.33:0/3234415537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 msgr2=0x5635d5dbdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 -- 172.21.15.33:0/3234415537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635d5cade70 con 0x5635d5d4eb80 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978939d640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9783fff640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d5da20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9770009850 tx=0x7f9770005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe4309e3cd745eda server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9763fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770005d50 con 0x5635d5d4eb80 2024-01-25T16:15:41.105 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97780ca310 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9770046040 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9770004dd0 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978d9c4740 10 client.25556 fetch_fsmap learned FSMap version 28 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.098+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5635d5c76a80 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97780ca310 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f977003c040 con 0x5635d5d4eb80 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 10 client.25556 fetch_fsmap finished waiting for FSMap version 28 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 20 client.25556 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5dbf8f0 0x5635d5dc1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 4 client.25556 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635d59b5600 con 0x5635d5dbf8f0 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5dbf8f0 0x5635d5dc1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f9763fff640 10 client.25556 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.099+0000 7f9788b9c640 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5dbf8f0 0x5635d5dc1d10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f87d4723efe0a2 server_cookie=d313c4632138d69e in_seq=0 out_seq=0 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f9763fff640 1 -- 172.21.15.33:0/3234415537 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635d59b5600 con 0x5635d5dbf8f0 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f9763fff640 10 client.25556 handle_command_reply: tid=0 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 client.25556 shutdown 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 msgr2=0x5635d5d5da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d5da20 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9770009850 tx=0x7f9770005750 comp rx=0 tx=0).stop 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 shutdown_connections 2024-01-25T16:15:41.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5635d5dbf8f0 0x5635d5dc1d10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f9763fff640 0 client.25556 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635d5d57950 0x5635d5dbdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5635d5d4eb80 0x5635d5d5da20 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 --2- 172.21.15.33:0/3234415537 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5635d5d4e260 0x5635d5d5d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 >> 172.21.15.33:0/3234415537 conn(0x5635d5d48b80 msgr2=0x5635d5d4a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 shutdown_connections 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 1 -- 172.21.15.33:0/3234415537 wait complete. 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.208+0000 7f978d9c4740 20 client.25556 trim_cache size 0 max 16384 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:15:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 msgr2=0x7f97740a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 0x7f97740a3900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f977c005fd0 tx=0x7f977c011810 comp rx=0 tx=0).stop 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 msgr2=0x7f9784199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f977800d770 tx=0x7f977800dc30 comp rx=0 tx=0).stop 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 shutdown_connections 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f97740a1440 0x7f97740a3900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f978419db90 0x7f978419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9784108040 0x7f9784199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 --2- 172.21.15.33:0/407921893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9784102f70 0x7f9784199320 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 >> 172.21.15.33:0/407921893 conn(0x7f97840fe7d0 msgr2=0x7f97840ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 shutdown_connections 2024-01-25T16:15:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:41.209+0000 7f9789b9e640 1 -- 172.21.15.33:0/407921893 wait complete. 2024-01-25T16:15:41.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:41 smithi033 ceph-mon[31417]: pgmap v244: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-01-25T16:15:41.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:41 smithi152 ceph-mon[36015]: pgmap v244: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-01-25T16:15:41.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:41 smithi076 ceph-mon[35642]: pgmap v244: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-01-25T16:15:41.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 422498 7.06 MB/sec execute 278 sec latency 104.573 ms 2024-01-25T16:15:42.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 424534 7.06 MB/sec execute 279 sec latency 20.455 ms 2024-01-25T16:15:43.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:43 smithi033 ceph-mon[31417]: pgmap v245: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 416 op/s 2024-01-25T16:15:43.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:43 smithi152 ceph-mon[36015]: pgmap v245: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 416 op/s 2024-01-25T16:15:43.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:43 smithi076 ceph-mon[35642]: pgmap v245: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.3 MiB/s wr, 416 op/s 2024-01-25T16:15:43.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 426464 7.08 MB/sec execute 280 sec latency 16.670 ms 2024-01-25T16:15:44.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 428456 7.09 MB/sec execute 281 sec latency 89.850 ms 2024-01-25T16:15:45.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:45 smithi033 ceph-mon[31417]: pgmap v246: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-01-25T16:15:45.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:45 smithi152 ceph-mon[36015]: pgmap v246: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-01-25T16:15:45.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:45 smithi076 ceph-mon[35642]: pgmap v246: 97 pgs: 97 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.8 MiB/s wr, 237 op/s 2024-01-25T16:15:45.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 430137 7.10 MB/sec execute 282 sec latency 94.854 ms 2024-01-25T16:15:46.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 432100 7.11 MB/sec execute 283 sec latency 215.959 ms 2024-01-25T16:15:47.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:47 smithi033 ceph-mon[31417]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 382 op/s 2024-01-25T16:15:47.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:47 smithi152 ceph-mon[36015]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 382 op/s 2024-01-25T16:15:47.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:47 smithi076 ceph-mon[35642]: pgmap v247: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.2 MiB/s wr, 382 op/s 2024-01-25T16:15:47.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 433815 7.12 MB/sec execute 284 sec latency 235.141 ms 2024-01-25T16:15:48.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 436382 7.15 MB/sec execute 285 sec latency 17.057 ms 2024-01-25T16:15:49.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:49 smithi033 ceph-mon[31417]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:49.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:49 smithi152 ceph-mon[36015]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:49.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:49 smithi076 ceph-mon[35642]: pgmap v248: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:49.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 437922 7.15 MB/sec execute 286 sec latency 48.707 ms 2024-01-25T16:15:50.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 440173 7.17 MB/sec execute 287 sec latency 84.050 ms 2024-01-25T16:15:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:51 smithi033 ceph-mon[31417]: pgmap v249: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:51.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:51 smithi152 ceph-mon[36015]: pgmap v249: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:51.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:51 smithi076 ceph-mon[35642]: pgmap v249: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:51.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 442225 7.17 MB/sec execute 288 sec latency 51.045 ms 2024-01-25T16:15:52.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 444195 7.19 MB/sec execute 289 sec latency 15.161 ms 2024-01-25T16:15:53.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:53 smithi033 ceph-mon[31417]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-01-25T16:15:53.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:53 smithi152 ceph-mon[36015]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-01-25T16:15:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:53 smithi076 ceph-mon[35642]: pgmap v250: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.7 MiB/s wr, 387 op/s 2024-01-25T16:15:53.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 446788 7.21 MB/sec execute 290 sec latency 13.675 ms 2024-01-25T16:15:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 -- 172.21.15.33:0/321316744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 msgr2=0x7f39401002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 --2- 172.21.15.33:0/321316744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f39401002c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3934009900 tx=0x7f393402bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 -- 172.21.15.33:0/321316744 shutdown_connections 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 --2- 172.21.15.33:0/321316744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 0x7f394010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 --2- 172.21.15.33:0/321316744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f39401002c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 --2- 172.21.15.33:0/321316744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 0x7f3940104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.317+0000 7f394509b640 1 -- 172.21.15.33:0/321316744 >> 172.21.15.33:0/321316744 conn(0x7f3940076a40 msgr2=0x7f3940076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.318+0000 7f394509b640 1 -- 172.21.15.33:0/321316744 shutdown_connections 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.318+0000 7f394509b640 1 -- 172.21.15.33:0/321316744 wait complete. 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.318+0000 7f394509b640 1 Processor -- start 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.318+0000 7f394509b640 1 -- start start 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 0x7f39401985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 0x7f39401a3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940198e30 con 0x7f39400ffe40 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401a3650 con 0x7f3940100800 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401a3790 con 0x7f3940104b90 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48822/0 (socket says 172.21.15.33:48822) 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 -- 172.21.15.33:0/260218513 learned_addr learned my addr 172.21.15.33:0/260218513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393e59c640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 0x7f39401985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393f59e640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 0x7f39401a3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 -- 172.21.15.33:0/260218513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 msgr2=0x7f39401a3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 0x7f39401a3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 -- 172.21.15.33:0/260218513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 msgr2=0x7f39401985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 0x7f39401985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 -- 172.21.15.33:0/260218513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3934009560 con 0x7f39400ffe40 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f393ed9d640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f393800d770 tx=0x7f393800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f3927fff640 1 -- 172.21.15.33:0/260218513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009c30 con 0x7f39400ffe40 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39401a39f0 con 0x7f39400ffe40 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f3927fff640 1 -- 172.21.15.33:0/260218513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f393800de80 con 0x7f39400ffe40 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.319+0000 7f3927fff640 1 -- 172.21.15.33:0/260218513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938004de0 con 0x7f39400ffe40 2024-01-25T16:15:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.320+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401a3e80 con 0x7f39400ffe40 2024-01-25T16:15:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3927fff640 1 -- 172.21.15.33:0/260218513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3938004f40 con 0x7f39400ffe40 2024-01-25T16:15:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 Processor -- start 2024-01-25T16:15:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3927fff640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 0x7f39280a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 -- start start 2024-01-25T16:15:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f393f59e640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 0x7f39280a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3927fff640 1 -- 172.21.15.33:0/260218513 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f39380cc910 con 0x7f39400ffe40 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8f8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f85e070 con 0x56504f8ef090 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f85b760 con 0x56504f8f8780 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.322+0000 7f3947c65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f857050 con 0x56504f8ef9b0 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48828/0 (socket says 172.21.15.33:48828) 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 -- 172.21.15.33:0/3190433140 learned_addr learned my addr 172.21.15.33:0/3190433140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393f59e640 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393e59c640 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 -- 172.21.15.33:0/3190433140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 msgr2=0x56504f8f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 -- 172.21.15.33:0/3190433140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 msgr2=0x56504f8fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 -- 172.21.15.33:0/3190433140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56504f84ee70 con 0x56504f8ef090 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393f59e640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 0x7f39280a39b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3930005fd0 tx=0x7f3930004410 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f393ed9d640 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3938095cb0 tx=0x7f39380b9050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74057f4259e329 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f39257fa640 1 -- 172.21.15.33:0/3190433140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938009d90 con 0x56504f8ef090 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f39257fa640 1 -- 172.21.15.33:0/3190433140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3938097b50 con 0x56504f8ef090 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.323+0000 7f39257fa640 1 -- 172.21.15.33:0/3190433140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938096080 con 0x56504f8ef090 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- 172.21.15.33:0/3190433140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 msgr2=0x56504f8ef470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3938095cb0 tx=0x7f39380b9050 comp rx=0 tx=0).stop 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- 172.21.15.33:0/3190433140 shutdown_connections 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 --2- 172.21.15.33:0/3190433140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ef470 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- 172.21.15.33:0/3190433140 >> 172.21.15.33:0/3190433140 conn(0x56504f8e99b0 msgr2=0x56504f8e9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- 172.21.15.33:0/3190433140 shutdown_connections 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- 172.21.15.33:0/3190433140 wait complete. 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 -- start start 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 Processor -- start 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.324+0000 7f3947c65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8f5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8101e0 con 0x56504f8ef9b0 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f80fe80 con 0x56504f8ef090 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f7d50c0 con 0x56504f8f8780 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 10 client.0 _ll_register_callbacks cb 0x56504f7faac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48836/0 (socket says 172.21.15.33:48836) 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 -- 172.21.15.33:0/767364032 learned_addr learned my addr 172.21.15.33:0/767364032 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393ed9d640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393f59e640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8f5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 -- 172.21.15.33:0/767364032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 msgr2=0x56504f8f5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8f5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 -- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 msgr2=0x56504f8f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 -- 172.21.15.33:0/767364032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401a3e80 con 0x56504f8ef9b0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393ed9d640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f393e59c640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f393402c170 tx=0x7f393402c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338fda12842d3909 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3922ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934004040 con 0x56504f8ef9b0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3934046070 con 0x56504f8ef9b0 2024-01-25T16:15:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56504f850040 con 0x56504f8ef9b0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39340041a0 con 0x56504f8ef9b0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 10 client.16047 fetch_fsmap learned FSMap version 28 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.325+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56504f8556d0 con 0x56504f8ef9b0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56504f850040 con 0x56504f8ef9b0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f393403c040 con 0x56504f8ef9b0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 10 client.16047 fetch_fsmap finished waiting for FSMap version 28 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 10 client.16047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 20 client.16047 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f9082f0 0x56504f90a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 4 client.16047 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56504f84fd60 con 0x56504f9082f0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f393ed9d640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f9082f0 0x56504f90a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f3922ff5640 10 client.16047 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.326+0000 7f393ed9d640 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f9082f0 0x56504f90a710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767e764df719de56 server_cookie=b9cc91506380fe8b in_seq=0 out_seq=0 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3922ff5640 1 -- 172.21.15.33:0/767364032 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56504f84fd60 con 0x56504f9082f0 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3922ff5640 10 client.16047 handle_command_reply: tid=0 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 client.16047 shutdown 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 msgr2=0x56504f8f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f393402c170 tx=0x7f393402c9e0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 shutdown_connections 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f9082f0 0x56504f90a710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.327+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8f5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8f55d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 1 --2- 172.21.15.33:0/767364032 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3922ff5640 0 client.16047 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 >> 172.21.15.33:0/767364032 conn(0x56504f8e99b0 msgr2=0x56504f8eb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.337 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 shutdown_connections 2024-01-25T16:15:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 1 -- 172.21.15.33:0/767364032 wait complete. 2024-01-25T16:15:54.338 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.328+0000 7f3947c65740 20 client.16047 trim_cache size 0 max 16384 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 Processor -- start 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 -- start start 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8fb030 con 0x56504f8ef9b0 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f90f880 con 0x56504f8ef090 2024-01-25T16:15:54.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.341+0000 7f3947c65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8e9fc0 con 0x56504f8f8780 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48842/0 (socket says 172.21.15.33:48842) 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 -- 172.21.15.33:0/2328272935 learned_addr learned my addr 172.21.15.33:0/2328272935 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393f59e640 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393ed9d640 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 -- 172.21.15.33:0/2328272935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 msgr2=0x56504f8ff8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8ff8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 -- 172.21.15.33:0/2328272935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 msgr2=0x56504f8f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 -- 172.21.15.33:0/2328272935 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56504f7f2060 con 0x56504f8ef9b0 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f393e59c640 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f393402cd60 tx=0x7f393402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d634cf44f1c66e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f3924ff9640 1 -- 172.21.15.33:0/2328272935 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934058070 con 0x56504f8ef9b0 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f3924ff9640 1 -- 172.21.15.33:0/2328272935 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3934044070 con 0x56504f8ef9b0 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.342+0000 7f3924ff9640 1 -- 172.21.15.33:0/2328272935 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934005000 con 0x56504f8ef9b0 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- 172.21.15.33:0/2328272935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 msgr2=0x56504f8ff3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f393402cd60 tx=0x7f393402bc30 comp rx=0 tx=0).stop 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- 172.21.15.33:0/2328272935 shutdown_connections 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8f8780 0x56504f8ff8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef9b0 0x56504f8ff3b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- 172.21.15.33:0/2328272935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8ef090 0x56504f8f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- 172.21.15.33:0/2328272935 >> 172.21.15.33:0/2328272935 conn(0x56504f8e99b0 msgr2=0x56504f8f4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- 172.21.15.33:0/2328272935 shutdown_connections 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- 172.21.15.33:0/2328272935 wait complete. 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- start start 2024-01-25T16:15:54.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 Processor -- start 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8ffc80 con 0x56504f8ef090 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8fe410 con 0x56504f8f8780 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f8fe580 con 0x56504f8ef9b0 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f3947c65740 10 client.0 _ll_register_callbacks cb 0x7f3934004dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f393ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48852/0 (socket says 172.21.15.33:48852) 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f393ed9d640 1 -- 172.21.15.33:0/1878126539 learned_addr learned my addr 172.21.15.33:0/1878126539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f393f59e640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.351 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.343+0000 7f393e59c640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 -- 172.21.15.33:0/1878126539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 msgr2=0x56504f8fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 -- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 msgr2=0x56504f8fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 -- 172.21.15.33:0/1878126539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56504f8556d0 con 0x56504f8ef090 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f393ed9d640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f39380ca100 tx=0x7f3938095af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32326014c9bb3bdc server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380b7070 con 0x56504f8ef090 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3938096010 con 0x56504f8ef090 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3938097b50 con 0x56504f8ef090 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39380ca2d0 con 0x56504f8ef090 2024-01-25T16:15:54.352 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 10 client.16053 fetch_fsmap learned FSMap version 28 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56504f817a80 con 0x56504f8ef090 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3938097b50 con 0x56504f8ef090 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39380a1600 con 0x56504f8ef090 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 10 client.16053 fetch_fsmap finished waiting for FSMap version 28 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 20 client.16053 populate_metadata read hostname 'smithi033' 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f95f810 0x56504f961c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 4 client.16053 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.344+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56504f556600 con 0x56504f95f810 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.345+0000 7f393e59c640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f95f810 0x56504f961c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.345+0000 7f3922ff5640 10 client.16053 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:54.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.345+0000 7f393e59c640 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f95f810 0x56504f961c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d22b61dcf78c0 server_cookie=8fd780f9db4d955b in_seq=0 out_seq=0 2024-01-25T16:15:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3922ff5640 1 -- 172.21.15.33:0/1878126539 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56504f556600 con 0x56504f95f810 2024-01-25T16:15:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3922ff5640 10 client.16053 handle_command_reply: tid=0 2024-01-25T16:15:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 client.16053 shutdown 2024-01-25T16:15:54.533 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 msgr2=0x56504f8ff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f39380ca100 tx=0x7f3938095af0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 shutdown_connections 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56504f95f810 0x56504f961c00 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56504f8f8780 0x56504f8fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56504f8ef9b0 0x56504f8fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 --2- 172.21.15.33:0/1878126539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56504f8ef090 0x56504f8ff3d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 >> 172.21.15.33:0/1878126539 conn(0x56504f8e99b0 msgr2=0x56504f8eb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3922ff5640 0 client.16053 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 shutdown_connections 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.530+0000 7f3947c65740 1 -- 172.21.15.33:0/1878126539 wait complete. 2024-01-25T16:15:54.534 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f3947c65740 20 client.16053 trim_cache size 0 max 16384 2024-01-25T16:15:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:15:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:15:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:15:54.535 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 msgr2=0x7f39280a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 0x7f39280a39b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3930005fd0 tx=0x7f3930004410 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 msgr2=0x7f3940198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f393800d770 tx=0x7f393800dc30 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 shutdown_connections 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f39280a14f0 0x7f39280a39b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3940104b90 0x7f39401a3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3940100800 0x7f39401985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 --2- 172.21.15.33:0/260218513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39400ffe40 0x7f3940198090 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 >> 172.21.15.33:0/260218513 conn(0x7f3940076a40 msgr2=0x7f39400fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 shutdown_connections 2024-01-25T16:15:54.536 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:15:54.531+0000 7f394509b640 1 -- 172.21.15.33:0/260218513 wait complete. 2024-01-25T16:15:54.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 448019 7.21 MB/sec execute 291 sec latency 67.096 ms 2024-01-25T16:15:55.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:55 smithi033 ceph-mon[31417]: pgmap v251: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-01-25T16:15:55.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:55 smithi152 ceph-mon[36015]: pgmap v251: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-01-25T16:15:55.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:55 smithi076 ceph-mon[35642]: pgmap v251: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-01-25T16:15:55.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 450454 7.24 MB/sec execute 292 sec latency 45.597 ms 2024-01-25T16:15:56.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 452104 7.24 MB/sec execute 293 sec latency 16.612 ms 2024-01-25T16:15:57.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:57 smithi033 ceph-mon[31417]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-01-25T16:15:57.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:57 smithi152 ceph-mon[36015]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-01-25T16:15:57.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:57 smithi076 ceph-mon[35642]: pgmap v252: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-01-25T16:15:57.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 454407 7.26 MB/sec execute 294 sec latency 14.357 ms 2024-01-25T16:15:58.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 456468 7.26 MB/sec execute 295 sec latency 21.713 ms 2024-01-25T16:15:59.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:15:59 smithi033 ceph-mon[31417]: pgmap v253: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:59.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:15:59 smithi152 ceph-mon[36015]: pgmap v253: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:59.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:15:59 smithi076 ceph-mon[35642]: pgmap v253: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:15:59.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458149 7.28 MB/sec execute 296 sec latency 80.446 ms 2024-01-25T16:16:00.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 459569 7.33 MB/sec execute 297 sec latency 61.463 ms 2024-01-25T16:16:01.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:01 smithi033 ceph-mon[31417]: pgmap v254: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:16:01.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:01 smithi152 ceph-mon[36015]: pgmap v254: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:16:01.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:01 smithi076 ceph-mon[35642]: pgmap v254: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.1 MiB/s wr, 283 op/s 2024-01-25T16:16:01.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 462117 7.36 MB/sec execute 298 sec latency 15.466 ms 2024-01-25T16:16:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:02 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:02 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:16:02.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:02 smithi033 ceph-mon[31417]: pgmap v255: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-01-25T16:16:02.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:02 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:02 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:16:02.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:02 smithi152 ceph-mon[36015]: pgmap v255: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-01-25T16:16:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:02 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:02 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:16:02.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:02 smithi076 ceph-mon[35642]: pgmap v255: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.6 MiB/s wr, 398 op/s 2024-01-25T16:16:02.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 463095 7.35 MB/sec execute 299 sec latency 29.264 ms 2024-01-25T16:16:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:03 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:03 smithi033 ceph-mon[31417]: osdmap e96: 12 total, 12 up, 12 in 2024-01-25T16:16:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:03 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:16:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:03 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:16:03.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:03 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:16:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:03 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:03 smithi152 ceph-mon[36015]: osdmap e96: 12 total, 12 up, 12 in 2024-01-25T16:16:03.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:03 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:16:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:03 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:16:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:03 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:16:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:03 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:03 smithi076 ceph-mon[35642]: osdmap e96: 12 total, 12 up, 12 in 2024-01-25T16:16:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:03 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:16:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:03 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:16:03.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:03 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:16:03.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 465561 7.37 MB/sec execute 300 sec latency 25.077 ms 2024-01-25T16:16:04.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:04 smithi033 ceph-mon[31417]: pgmap v257: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-01-25T16:16:04.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:04 smithi152 ceph-mon[36015]: pgmap v257: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-01-25T16:16:04.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:04 smithi076 ceph-mon[35642]: pgmap v257: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 354 op/s 2024-01-25T16:16:04.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 467114 7.37 MB/sec execute 301 sec latency 89.428 ms 2024-01-25T16:16:05.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469440 7.39 MB/sec execute 302 sec latency 17.924 ms 2024-01-25T16:16:06.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 471715 7.40 MB/sec execute 303 sec latency 15.408 ms 2024-01-25T16:16:07.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:07 smithi033 ceph-mon[31417]: pgmap v258: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 -- 172.21.15.33:0/1653167269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 msgr2=0x7fd3e41002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 --2- 172.21.15.33:0/1653167269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e41002c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd3d4009900 tx=0x7fd3d402bc30 comp rx=0 tx=0).stop 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 -- 172.21.15.33:0/1653167269 shutdown_connections 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 --2- 172.21.15.33:0/1653167269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 --2- 172.21.15.33:0/1653167269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e41002c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 --2- 172.21.15.33:0/1653167269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 0x7fd3e4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 -- 172.21.15.33:0/1653167269 >> 172.21.15.33:0/1653167269 conn(0x7fd3e4076a40 msgr2=0x7fd3e4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 -- 172.21.15.33:0/1653167269 shutdown_connections 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.626+0000 7fd3eb456640 1 -- 172.21.15.33:0/1653167269 wait complete. 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 Processor -- start 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 -- start start 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e4197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 0x7fd3e419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419c0b0 con 0x7fd3e40ffe40 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419c220 con 0x7fd3e4104b90 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3eb456640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e419c3e0 con 0x7fd3e4100800 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46652/0 (socket says 172.21.15.33:46652) 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 -- 172.21.15.33:0/861991847 learned_addr learned my addr 172.21.15.33:0/861991847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 0x7fd3e419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e4197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 -- 172.21.15.33:0/861991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 msgr2=0x7fd3e4197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e4197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 -- 172.21.15.33:0/861991847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 msgr2=0x7fd3e419bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 0x7fd3e419bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 -- 172.21.15.33:0/861991847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d4009560 con 0x7fd3e40ffe40 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e4197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.627+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e9f0 tx=0x7fd3e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.628+0000 7fd3da7fc640 1 -- 172.21.15.33:0/861991847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0009700 con 0x7fd3e40ffe40 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.628+0000 7fd3da7fc640 1 -- 172.21.15.33:0/861991847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e000cd80 con 0x7fd3e40ffe40 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.628+0000 7fd3da7fc640 1 -- 172.21.15.33:0/861991847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e0018860 con 0x7fd3e40ffe40 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.628+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e419c6c0 con 0x7fd3e40ffe40 2024-01-25T16:16:07.634 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.628+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e41a3f40 con 0x7fd3e40ffe40 2024-01-25T16:16:07.635 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:07 smithi152 ceph-mon[36015]: pgmap v258: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:07.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.630+0000 7fd3da7fc640 1 -- 172.21.15.33:0/861991847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd3e00189c0 con 0x7fd3e40ffe40 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.630+0000 7fd3ee011740 1 Processor -- start 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 -- start start 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3da7fc640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 0x7fd3d00a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c7964f070 con 0x563c796e9780 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c7964c760 con 0x563c796e09b0 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3ee011740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c79648050 con 0x563c796e0090 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3da7fc640 1 -- 172.21.15.33:0/861991847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd3e0014070 con 0x7fd3e40ffe40 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 0x7fd3d00a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e89f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:40568/0 (socket says 172.21.15.33:40568) 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 -- 172.21.15.33:0/751008430 learned_addr learned my addr 172.21.15.33:0/751008430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 -- 172.21.15.33:0/751008430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 msgr2=0x563c796e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 -- 172.21.15.33:0/751008430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 msgr2=0x563c796ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e91f2640 1 -- 172.21.15.33:0/751008430 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c7963fe70 con 0x563c796e0090 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.631+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0096590 tx=0x7fd3e0096500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1de7d09e44c3c9a5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 0x7fd3d00a3a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005fd0 tx=0x7fd3dc013b60 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3cf7fe640 1 -- 172.21.15.33:0/751008430 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00cc040 con 0x563c796e0090 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3cf7fe640 1 -- 172.21.15.33:0/751008430 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3e009a090 con 0x563c796e0090 2024-01-25T16:16:07.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3cf7fe640 1 -- 172.21.15.33:0/751008430 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e00589f0 con 0x563c796e0090 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3ee011740 1 -- 172.21.15.33:0/751008430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 msgr2=0x563c796e0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.632+0000 7fd3ee011740 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0096590 tx=0x7fd3e0096500 comp rx=0 tx=0).stop 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- 172.21.15.33:0/751008430 shutdown_connections 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- 172.21.15.33:0/751008430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e0470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- 172.21.15.33:0/751008430 >> 172.21.15.33:0/751008430 conn(0x563c796da9b0 msgr2=0x563c796dad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- 172.21.15.33:0/751008430 shutdown_connections 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- 172.21.15.33:0/751008430 wait complete. 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- start start 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 Processor -- start 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796011e0 con 0x563c796e9780 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c79600e80 con 0x563c796e09b0 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.633+0000 7fd3ee011740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c795c60c0 con 0x563c796e0090 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ee011740 10 client.0 _ll_register_callbacks cb 0x563c795ebac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46660/0 (socket says 172.21.15.33:46660) 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e89f1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 -- 172.21.15.33:0/4173225266 learned_addr learned my addr 172.21.15.33:0/4173225266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 -- 172.21.15.33:0/4173225266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 msgr2=0x563c796e6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 -- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 msgr2=0x563c796e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 -- 172.21.15.33:0/4173225266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e41a3f40 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00fd70 tx=0x7fd3dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210a75eae75dba79 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc004b20 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd3dc004c80 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c79641040 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc004e40 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ee011740 10 client.16065 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.634+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c796466d0 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563c79641040 con 0x563c796e9780 2024-01-25T16:16:07.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd3dc0205b0 con 0x563c796e9780 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 10 client.16065 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 20 client.16065 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796f9600 0x563c796fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 4 client.16065 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c79640d60 con 0x563c796f9600 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.635+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796f9600 0x563c796fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.636+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796f9600 0x563c796fba20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9308b1d17383fd6c server_cookie=6bcc92765816b093 in_seq=0 out_seq=0 2024-01-25T16:16:07.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.636+0000 7fd3ccff9640 10 client.16065 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.637+0000 7fd3ccff9640 1 -- 172.21.15.33:0/4173225266 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c79640d60 con 0x563c796f9600 2024-01-25T16:16:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.637+0000 7fd3ccff9640 10 client.16065 handle_command_reply: tid=0 2024-01-25T16:16:07.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.637+0000 7fd3ee011740 1 client.16065 shutdown 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 msgr2=0x563c796e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc00fd70 tx=0x7fd3dc004ab0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 shutdown_connections 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796f9600 0x563c796fba20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e9780 0x563c796e71f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 --2- 172.21.15.33:0/4173225266 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ccff9640 0 client.16065 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 >> 172.21.15.33:0/4173225266 conn(0x563c796da9b0 msgr2=0x563c796dc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 shutdown_connections 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 1 -- 172.21.15.33:0/4173225266 wait complete. 2024-01-25T16:16:07.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.638+0000 7fd3ee011740 20 client.16065 trim_cache size 0 max 16384 2024-01-25T16:16:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 Processor -- start 2024-01-25T16:16:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 -- start start 2024-01-25T16:16:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796f7d90 con 0x563c796da9b0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796f6050 con 0x563c796e09b0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3ee011740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796da4e0 con 0x563c796e0090 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3e99f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3e91f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46672/0 (socket says 172.21.15.33:46672) 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.652+0000 7fd3e91f2640 1 -- 172.21.15.33:0/3572426391 learned_addr learned my addr 172.21.15.33:0/3572426391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3572426391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 msgr2=0x563c796e7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3572426391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 msgr2=0x563c796e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3572426391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c795e3060 con 0x563c796e09b0 2024-01-25T16:16:07.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e39d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012040 tx=0x7fd3dc009af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159dedae1047161f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3ceffd640 1 -- 172.21.15.33:0/3572426391 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc009ec0 con 0x563c796e09b0 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3ceffd640 1 -- 172.21.15.33:0/3572426391 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3dc005b70 con 0x563c796e09b0 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.653+0000 7fd3ceffd640 1 -- 172.21.15.33:0/3572426391 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc029540 con 0x563c796e09b0 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- 172.21.15.33:0/3572426391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 msgr2=0x563c796e39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e39d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012040 tx=0x7fd3dc009af0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- 172.21.15.33:0/3572426391 shutdown_connections 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796e09b0 0x563c796e39d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796e7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3572426391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796da9b0 0x563c796e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- 172.21.15.33:0/3572426391 >> 172.21.15.33:0/3572426391 conn(0x563c796e13e0 msgr2=0x563c79700360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- 172.21.15.33:0/3572426391 shutdown_connections 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- 172.21.15.33:0/3572426391 wait complete. 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 -- start start 2024-01-25T16:16:07.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.654+0000 7fd3ee011740 1 Processor -- start 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 0x563c796ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796ef610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796efde0 con 0x563c796e09b0 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c796eff50 con 0x563c796da9b0 2024-01-25T16:16:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c79750060 con 0x563c796e0090 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 10 client.0 _ll_register_callbacks cb 0x7fd3e000cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46680/0 (socket says 172.21.15.33:46680) 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3984687956 learned_addr learned my addr 172.21.15.33:0/3984687956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 0x563c796ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e89f1640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3984687956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 msgr2=0x563c796ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 msgr2=0x563c796ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 0x563c796ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 -- 172.21.15.33:0/3984687956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c7963fe70 con 0x563c796e09b0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 0x563c796ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3e99f3640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012420 tx=0x7fd3dc00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad23f8d374cd91a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc009ec0 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c79347600 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.655+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd3dc020430 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3dc005b50 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 10 client.16071 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x563c796466d0 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3dc049080 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7fd3dc03b020 con 0x563c796e09b0 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 10 client.16071 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 20 client.16071 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796e9780 0x563c797535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 4 client.16071 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563c7939ac50 con 0x563c796e9780 2024-01-25T16:16:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.656+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796e9780 0x563c797535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.657+0000 7fd3ccff9640 10 client.16071 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:07.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.657+0000 7fd3e91f2640 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796e9780 0x563c797535a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c71574ff872f504 server_cookie=b8c49271ffc575ab in_seq=0 out_seq=0 2024-01-25T16:16:07.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:07 smithi076 ceph-mon[35642]: pgmap v258: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:07.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 473647 7.41 MB/sec execute 304 sec latency 59.179 ms 2024-01-25T16:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ccff9640 1 -- 172.21.15.33:0/3984687956 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563c7939ac50 con 0x563c796e9780 2024-01-25T16:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ccff9640 10 client.16071 handle_command_reply: tid=0 2024-01-25T16:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ee011740 1 client.16071 shutdown 2024-01-25T16:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 msgr2=0x563c7974faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc012420 tx=0x7fd3dc00f8f0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 shutdown_connections 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.802+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563c796e9780 0x563c797535a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ccff9640 0 client.16071 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c796e09b0 0x563c7974faf0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563c796e0090 0x563c796ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 --2- 172.21.15.33:0/3984687956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c796da9b0 0x563c796ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 >> 172.21.15.33:0/3984687956 conn(0x563c796e13e0 msgr2=0x563c796d91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 shutdown_connections 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 1 -- 172.21.15.33:0/3984687956 wait complete. 2024-01-25T16:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3ee011740 20 client.16071 trim_cache size 0 max 16384 2024-01-25T16:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.803+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 msgr2=0x7fd3d00a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 0x7fd3d00a3a30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc005fd0 tx=0x7fd3dc013b60 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 msgr2=0x7fd3e4197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e9f0 tx=0x7fd3e000eeb0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 shutdown_connections 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd3d00a1570 0x7fd3d00a3a30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3e4104b90 0x7fd3e419bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd3e4100800 0x7fd3e4197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 --2- 172.21.15.33:0/861991847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3e40ffe40 0x7fd3e4197240 secure :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd3e000e9f0 tx=0x7fd3e000eeb0 comp rx=0 tx=0).stop 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 >> 172.21.15.33:0/861991847 conn(0x7fd3e4076a40 msgr2=0x7fd3e40685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 shutdown_connections 2024-01-25T16:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:07.804+0000 7fd3eb456640 1 -- 172.21.15.33:0/861991847 wait complete. 2024-01-25T16:16:08.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:08 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:08.636 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:08 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:08.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:08 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:08.780 INFO:tasks.workunit.client.0.smithi033.stdout: 1 475723 7.43 MB/sec execute 305 sec latency 201.655 ms 2024-01-25T16:16:09.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:09 smithi033 ceph-mon[31417]: pgmap v259: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:09.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:09 smithi152 ceph-mon[36015]: pgmap v259: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:09.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:09 smithi076 ceph-mon[35642]: pgmap v259: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:09.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 477223 7.43 MB/sec execute 306 sec latency 86.843 ms 2024-01-25T16:16:10.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479136 7.44 MB/sec execute 307 sec latency 160.566 ms 2024-01-25T16:16:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:11 smithi033 ceph-mon[31417]: pgmap v260: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:11.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:11 smithi033 ceph-mon[31417]: osdmap e97: 12 total, 12 up, 12 in 2024-01-25T16:16:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:11 smithi152 ceph-mon[36015]: pgmap v260: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:11.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:11 smithi152 ceph-mon[36015]: osdmap e97: 12 total, 12 up, 12 in 2024-01-25T16:16:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:11 smithi076 ceph-mon[35642]: pgmap v260: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 260 op/s 2024-01-25T16:16:11.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:11 smithi076 ceph-mon[35642]: osdmap e97: 12 total, 12 up, 12 in 2024-01-25T16:16:11.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 481227 7.45 MB/sec execute 308 sec latency 15.779 ms 2024-01-25T16:16:12.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483638 7.47 MB/sec execute 309 sec latency 13.133 ms 2024-01-25T16:16:13.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:13 smithi033 ceph-mon[31417]: pgmap v262: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-01-25T16:16:13.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:13 smithi152 ceph-mon[36015]: pgmap v262: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-01-25T16:16:13.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:13 smithi076 ceph-mon[35642]: pgmap v262: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-01-25T16:16:13.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 485709 7.47 MB/sec execute 310 sec latency 46.806 ms 2024-01-25T16:16:14.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 487760 7.49 MB/sec execute 311 sec latency 51.333 ms 2024-01-25T16:16:15.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:15 smithi033 ceph-mon[31417]: pgmap v263: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 362 op/s 2024-01-25T16:16:15.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:15 smithi152 ceph-mon[36015]: pgmap v263: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 362 op/s 2024-01-25T16:16:15.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:15 smithi076 ceph-mon[35642]: pgmap v263: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.8 MiB/s wr, 362 op/s 2024-01-25T16:16:15.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490102 7.51 MB/sec execute 312 sec latency 39.139 ms 2024-01-25T16:16:16.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 492003 7.52 MB/sec execute 313 sec latency 19.330 ms 2024-01-25T16:16:17.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:17 smithi033 ceph-mon[31417]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:17.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:17 smithi152 ceph-mon[36015]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:17.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:17 smithi076 ceph-mon[35642]: pgmap v264: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:17.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 494243 7.54 MB/sec execute 314 sec latency 87.793 ms 2024-01-25T16:16:18.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 496427 7.54 MB/sec execute 315 sec latency 15.940 ms 2024-01-25T16:16:19.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:19 smithi033 ceph-mon[31417]: pgmap v265: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:19.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:19 smithi152 ceph-mon[36015]: pgmap v265: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:19 smithi076 ceph-mon[35642]: pgmap v265: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:19.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 498458 7.55 MB/sec execute 316 sec latency 18.575 ms 2024-01-25T16:16:20.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 500567 7.57 MB/sec execute 317 sec latency 74.610 ms 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 -- 172.21.15.33:0/1024278751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36600ffe40 msgr2=0x7f36601002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1024278751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36600ffe40 0x7f36601002a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f364c009900 tx=0x7f364c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 -- 172.21.15.33:0/1024278751 shutdown_connections 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1024278751 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36601007e0 0x7f366010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1024278751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36600ffe40 0x7f36601002a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1024278751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 0x7f3660104f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 -- 172.21.15.33:0/1024278751 >> 172.21.15.33:0/1024278751 conn(0x7f36600769c0 msgr2=0x7f3660076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 -- 172.21.15.33:0/1024278751 shutdown_connections 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.897+0000 7f3666e1e640 1 -- 172.21.15.33:0/1024278751 wait complete. 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 Processor -- start 2024-01-25T16:16:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 -- start start 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36600ffe40 0x7f36600fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 0x7f36600fe370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a55a0 con 0x7f36601007e0 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a5710 con 0x7f3660104b70 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3666e1e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36601a58d0 con 0x7f36600ffe40 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58634/0 (socket says 172.21.15.33:58634) 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f365ffff640 1 -- 172.21.15.33:0/1395472576 learned_addr learned my addr 172.21.15.33:0/1395472576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f36653bb640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 0x7f36600fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.898+0000 7f3664bba640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36600ffe40 0x7f36600fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 -- 172.21.15.33:0/1395472576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36600ffe40 msgr2=0x7f36600fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36600ffe40 0x7f36600fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 -- 172.21.15.33:0/1395472576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 msgr2=0x7f36600fe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 0x7f36600fe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 -- 172.21.15.33:0/1395472576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c009560 con 0x7f36601007e0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365ffff640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365dffb640 1 -- 172.21.15.33:0/1395472576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c004250 con 0x7f36601007e0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36601a5b50 con 0x7f36601007e0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365dffb640 1 -- 172.21.15.33:0/1395472576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f364c0043b0 con 0x7f36601007e0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f365dffb640 1 -- 172.21.15.33:0/1395472576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c003c00 con 0x7f36601007e0 2024-01-25T16:16:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.899+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36601a60b0 con 0x7f36601007e0 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.901+0000 7f365dffb640 1 -- 172.21.15.33:0/1395472576 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f364c005000 con 0x7f36601007e0 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 Processor -- start 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 -- start start 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc758070 con 0x55d2bc7e99b0 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc755760 con 0x55d2bc7e9090 2024-01-25T16:16:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36699e7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc751050 con 0x55d2bc7f2780 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f365dffb640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 0x7f36500a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58640/0 (socket says 172.21.15.33:58640) 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 -- 172.21.15.33:0/3056857587 learned_addr learned my addr 172.21.15.33:0/3056857587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36653bb640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36653bb640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 0x7f36500a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f365ffff640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f365dffb640 1 -- 172.21.15.33:0/1395472576 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f364c0ef6c0 con 0x7f36601007e0 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 -- 172.21.15.33:0/3056857587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 msgr2=0x55d2bc7f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 -- 172.21.15.33:0/3056857587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 msgr2=0x55d2bc7e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f3664bba640 1 -- 172.21.15.33:0/3056857587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2bc748e70 con 0x55d2bc7e99b0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.902+0000 7f36653bb640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f3664bba640 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3654009c70 tx=0x7f3654011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4d6a13f7abb212 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36653bb640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 0x7f36500a39e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3658009fd0 tx=0x7f3658009600 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f364affd640 1 -- 172.21.15.33:0/3056857587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654004230 con 0x55d2bc7e99b0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f364affd640 1 -- 172.21.15.33:0/3056857587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3654004390 con 0x55d2bc7e99b0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f364affd640 1 -- 172.21.15.33:0/3056857587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654019760 con 0x55d2bc7e99b0 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 -- 172.21.15.33:0/3056857587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 msgr2=0x55d2bc7f2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3654009c70 tx=0x7f3654011810 comp rx=0 tx=0).stop 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 -- 172.21.15.33:0/3056857587 shutdown_connections 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f2240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 --2- 172.21.15.33:0/3056857587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 -- 172.21.15.33:0/3056857587 >> 172.21.15.33:0/3056857587 conn(0x55d2bc7e39b0 msgr2=0x55d2bc7e3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 -- 172.21.15.33:0/3056857587 shutdown_connections 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.903+0000 7f36699e7740 1 -- 172.21.15.33:0/3056857587 wait complete. 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 -- start start 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 Processor -- start 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 0x55d2bc7ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc70a1e0 con 0x55d2bc7e9090 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc709e80 con 0x55d2bc7e99b0 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc6cf0c0 con 0x55d2bc7f2780 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f36699e7740 10 client.0 _ll_register_callbacks cb 0x55d2bc6f4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.904+0000 7f3664bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 0x55d2bc7ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f365ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f3664bba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 0x55d2bc7ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58652/0 (socket says 172.21.15.33:58652) 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f3664bba640 1 -- 172.21.15.33:0/1803417385 learned_addr learned my addr 172.21.15.33:0/1803417385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 -- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 msgr2=0x55d2bc7efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 -- 172.21.15.33:0/1803417385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 msgr2=0x55d2bc7ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 0x55d2bc7ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 -- 172.21.15.33:0/1803417385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36540098d0 con 0x55d2bc7f2780 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f365ffff640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36653bb640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f00f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3658012fe0 tx=0x7f3658004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b3b9af6bf9e505 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f363ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658011c00 con 0x55d2bc7f2780 2024-01-25T16:16:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2bc74a040 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3658011d60 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.905+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3658004b20 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 10 client.25636 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d2bc74f6d0 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2bc74a040 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3658021d10 con 0x55d2bc7f2780 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 10 client.25636 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 10 client.25636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 20 client.25636 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc8025f0 0x55d2bc804a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 4 client.25636 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2bc749d60 con 0x55d2bc8025f0 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f3664bba640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc8025f0 0x55d2bc804a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f363ffff640 10 client.25636 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.906+0000 7f3664bba640 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc8025f0 0x55d2bc804a10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f697787ecc328d server_cookie=aefdfe650e9bf8e2 in_seq=0 out_seq=0 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.908+0000 7f363ffff640 1 -- 172.21.15.33:0/1803417385 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2bc749d60 con 0x55d2bc8025f0 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.908+0000 7f363ffff640 10 client.25636 handle_command_reply: tid=0 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 client.25636 shutdown 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 msgr2=0x55d2bc7f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f00f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3658012fe0 tx=0x7f3658004a40 comp rx=0 tx=0).stop 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 shutdown_connections 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc8025f0 0x55d2bc804a10 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f00f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 --2- 172.21.15.33:0/1803417385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e9090 0x55d2bc7ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f363ffff640 0 client.25636 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 >> 172.21.15.33:0/1803417385 conn(0x55d2bc7e39b0 msgr2=0x55d2bc7e5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 shutdown_connections 2024-01-25T16:16:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 1 -- 172.21.15.33:0/1803417385 wait complete. 2024-01-25T16:16:20.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.909+0000 7f36699e7740 20 client.25636 trim_cache size 0 max 16384 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.923+0000 7f36699e7740 1 Processor -- start 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.923+0000 7f36699e7740 1 -- start start 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc74a040 con 0x55d2bc7e99b0 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc7f4f30 con 0x55d2bc7e9090 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36699e7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc7e49b0 con 0x55d2bc7f2780 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58656/0 (socket says 172.21.15.33:58656) 2024-01-25T16:16:20.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 -- 172.21.15.33:0/952821749 learned_addr learned my addr 172.21.15.33:0/952821749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f3664bba640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f36653bb640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 -- 172.21.15.33:0/952821749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 msgr2=0x55d2bc7ec8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7ec8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 -- 172.21.15.33:0/952821749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 msgr2=0x55d2bc7f1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 -- 172.21.15.33:0/952821749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2bc6ec060 con 0x55d2bc7e99b0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f3664bba640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f365ffff640 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f364c0bc4c0 tx=0x7f364c04ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee232644d2c2255a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f364a7fc640 1 -- 172.21.15.33:0/952821749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0bbe90 con 0x55d2bc7e99b0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f364a7fc640 1 -- 172.21.15.33:0/952821749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f364c0c13c0 con 0x55d2bc7e99b0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.924+0000 7f364a7fc640 1 -- 172.21.15.33:0/952821749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0c48c0 con 0x55d2bc7e99b0 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 -- 172.21.15.33:0/952821749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 msgr2=0x55d2bc7eff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f364c0bc4c0 tx=0x7f364c04ef50 comp rx=0 tx=0).stop 2024-01-25T16:16:20.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 -- 172.21.15.33:0/952821749 shutdown_connections 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7ec8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7eff10 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 --2- 172.21.15.33:0/952821749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 -- 172.21.15.33:0/952821749 >> 172.21.15.33:0/952821749 conn(0x55d2bc7e39b0 msgr2=0x55d2bc7edf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 -- 172.21.15.33:0/952821749 shutdown_connections 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.925+0000 7f36699e7740 1 -- 172.21.15.33:0/952821749 wait complete. 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 -- start start 2024-01-25T16:16:20.930 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 Processor -- start 2024-01-25T16:16:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc7f9dc0 con 0x55d2bc7e99b0 2024-01-25T16:16:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc858b20 con 0x55d2bc7e9090 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2bc858c90 con 0x55d2bc7f2780 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36699e7740 10 client.0 _ll_register_callbacks cb 0x7f36540097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f3664bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f36653bb640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f3664bba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:47134/0 (socket says 172.21.15.33:47134) 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f3664bba640 1 -- 172.21.15.33:0/2295886735 learned_addr learned my addr 172.21.15.33:0/2295886735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.926+0000 7f365ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58662/0 (socket says 172.21.15.33:58662) 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 -- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 msgr2=0x55d2bc7f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 -- 172.21.15.33:0/2295886735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 msgr2=0x55d2bc7f84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 -- 172.21.15.33:0/2295886735 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f365401e040 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f3664bba640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f365ffff640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36653bb640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f89e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3658006010 tx=0x7f3658020750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2eae6d54c28bca server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658011c00 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3654004390 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3658011d60 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3658020550 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36699e7740 10 client.25642 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f36601a60b0 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3654004390 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.927+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3658021480 con 0x55d2bc7f2780 2024-01-25T16:16:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 10 client.25642 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 20 client.25642 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc85a510 0x55d2bc85c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 4 client.25642 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d2bc450600 con 0x55d2bc85a510 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f3664bba640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc85a510 0x55d2bc85c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f3664bba640 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc85a510 0x55d2bc85c930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8e1f472518bc44 server_cookie=bd83cd3aba2153a4 in_seq=0 out_seq=0 2024-01-25T16:16:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:20.928+0000 7f363ffff640 10 client.25642 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.036+0000 7f363ffff640 1 -- 172.21.15.33:0/2295886735 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d2bc450600 con 0x55d2bc85a510 2024-01-25T16:16:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.036+0000 7f363ffff640 10 client.25642 handle_command_reply: tid=0 2024-01-25T16:16:21.039 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 client.25642 shutdown 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 msgr2=0x55d2bc7f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f89e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3658006010 tx=0x7f3658020750 comp rx=0 tx=0).stop 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 shutdown_connections 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d2bc85a510 0x55d2bc85c930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d2bc7f2780 0x55d2bc7f89e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2bc7e99b0 0x55d2bc7f84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f363ffff640 0 client.25642 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 --2- 172.21.15.33:0/2295886735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d2bc7e9090 0x55d2bc7f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 >> 172.21.15.33:0/2295886735 conn(0x55d2bc7e39b0 msgr2=0x55d2bc7e21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 shutdown_connections 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 1 -- 172.21.15.33:0/2295886735 wait complete. 2024-01-25T16:16:21.040 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.037+0000 7f36699e7740 20 client.25642 trim_cache size 0 max 16384 2024-01-25T16:16:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:16:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:16:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:16:21.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 msgr2=0x7f36500a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 0x7f36500a39e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3658009fd0 tx=0x7f3658009600 comp rx=0 tx=0).stop 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 msgr2=0x7f36600fde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f364c006010 tx=0x7f364c005e00 comp rx=0 tx=0).stop 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 shutdown_connections 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f36500a1520 0x7f36500a39e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3660104b70 0x7f36600fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36601007e0 0x7f36600fde30 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 --2- 172.21.15.33:0/1395472576 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f36600ffe40 0x7f36600fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 >> 172.21.15.33:0/1395472576 conn(0x7f36600769c0 msgr2=0x7f36600697f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 shutdown_connections 2024-01-25T16:16:21.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:21.038+0000 7f3666e1e640 1 -- 172.21.15.33:0/1395472576 wait complete. 2024-01-25T16:16:21.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:21 smithi033 ceph-mon[31417]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:21.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:21 smithi152 ceph-mon[36015]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:21.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:21 smithi076 ceph-mon[35642]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-01-25T16:16:21.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 502341 7.58 MB/sec execute 318 sec latency 40.643 ms 2024-01-25T16:16:22.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 504736 7.60 MB/sec execute 319 sec latency 16.584 ms 2024-01-25T16:16:23.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:23 smithi033 ceph-mon[31417]: pgmap v267: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-01-25T16:16:23.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:23 smithi152 ceph-mon[36015]: pgmap v267: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-01-25T16:16:23.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:23 smithi076 ceph-mon[35642]: pgmap v267: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 468 op/s 2024-01-25T16:16:23.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 506868 7.60 MB/sec execute 320 sec latency 15.942 ms 2024-01-25T16:16:24.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508840 7.61 MB/sec execute 321 sec latency 19.627 ms 2024-01-25T16:16:25.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:25 smithi033 ceph-mon[31417]: pgmap v268: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-01-25T16:16:25.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:25 smithi152 ceph-mon[36015]: pgmap v268: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-01-25T16:16:25.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:25 smithi076 ceph-mon[35642]: pgmap v268: 97 pgs: 97 active+clean; 79 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 268 op/s 2024-01-25T16:16:25.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 510476 7.61 MB/sec execute 322 sec latency 179.020 ms 2024-01-25T16:16:26.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 512716 7.63 MB/sec execute 323 sec latency 15.463 ms 2024-01-25T16:16:27.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:27 smithi033 ceph-mon[31417]: pgmap v269: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 364 op/s 2024-01-25T16:16:27.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:27 smithi152 ceph-mon[36015]: pgmap v269: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 364 op/s 2024-01-25T16:16:27.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:27 smithi076 ceph-mon[35642]: pgmap v269: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 364 op/s 2024-01-25T16:16:27.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515360 7.66 MB/sec execute 324 sec latency 15.283 ms 2024-01-25T16:16:28.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:28 smithi033 ceph-mon[31417]: pgmap v270: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:28.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:28 smithi152 ceph-mon[36015]: pgmap v270: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:28.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:28 smithi076 ceph-mon[35642]: pgmap v270: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:28.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 517514 7.66 MB/sec execute 325 sec latency 16.950 ms 2024-01-25T16:16:29.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 519556 7.67 MB/sec execute 326 sec latency 15.865 ms 2024-01-25T16:16:30.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 521867 7.69 MB/sec execute 327 sec latency 37.323 ms 2024-01-25T16:16:31.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:31 smithi033 ceph-mon[31417]: pgmap v271: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:31.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:31 smithi152 ceph-mon[36015]: pgmap v271: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:31.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:31 smithi076 ceph-mon[35642]: pgmap v271: 97 pgs: 97 active+clean; 87 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 3.0 MiB/s wr, 263 op/s 2024-01-25T16:16:31.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 523873 7.70 MB/sec execute 328 sec latency 16.462 ms 2024-01-25T16:16:32.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 526196 7.72 MB/sec execute 329 sec latency 38.905 ms 2024-01-25T16:16:33.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:33 smithi033 ceph-mon[31417]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 365 op/s 2024-01-25T16:16:33.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:33 smithi152 ceph-mon[36015]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 365 op/s 2024-01-25T16:16:33.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:33 smithi076 ceph-mon[35642]: pgmap v272: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.7 MiB/s wr, 365 op/s 2024-01-25T16:16:33.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 528513 7.73 MB/sec execute 330 sec latency 16.706 ms 2024-01-25T16:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 -- 172.21.15.33:0/38855115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 msgr2=0x7f89f40ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/38855115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f40ff240 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009900 tx=0x7f89e402bc30 comp rx=0 tx=0).stop 2024-01-25T16:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 -- 172.21.15.33:0/38855115 shutdown_connections 2024-01-25T16:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/38855115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f40ff780 0x7f89f410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/38855115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f40ff240 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/38855115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f4106dd0 0x7f89f41071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 -- 172.21.15.33:0/38855115 >> 172.21.15.33:0/38855115 conn(0x7f89f4075d80 msgr2=0x7f89f40fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 -- 172.21.15.33:0/38855115 shutdown_connections 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.128+0000 7f89fb5ec640 1 -- 172.21.15.33:0/38855115 wait complete. 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 Processor -- start 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 -- start start 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f40ff780 0x7f89f4197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f4106dd0 0x7f89f419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419bfd0 con 0x7f89f40fedc0 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419c140 con 0x7f89f4106dd0 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89fb5ec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419c300 con 0x7f89f40ff780 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34580/0 (socket says 172.21.15.33:34580) 2024-01-25T16:16:34.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 -- 172.21.15.33:0/236278097 learned_addr learned my addr 172.21.15.33:0/236278097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9b89640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f4106dd0 0x7f89f419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f8b87640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f40ff780 0x7f89f4197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 -- 172.21.15.33:0/236278097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f40ff780 msgr2=0x7f89f4197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f40ff780 0x7f89f4197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 -- 172.21.15.33:0/236278097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f4106dd0 msgr2=0x7f89f419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f4106dd0 0x7f89f419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.129+0000 7f89f9388640 1 -- 172.21.15.33:0/236278097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009560 con 0x7f89f40fedc0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89f9388640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e9f0 tx=0x7f89f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89ea7fc640 1 -- 172.21.15.33:0/236278097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0009720 con 0x7f89f40fedc0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f419c5e0 con 0x7f89f40fedc0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89ea7fc640 1 -- 172.21.15.33:0/236278097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89f000cd80 con 0x7f89f40fedc0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89ea7fc640 1 -- 172.21.15.33:0/236278097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0010430 con 0x7f89f40fedc0 2024-01-25T16:16:34.134 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.130+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f41a3e60 con 0x7f89f40fedc0 2024-01-25T16:16:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89ea7fc640 1 -- 172.21.15.33:0/236278097 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f89f0005020 con 0x7f89f40fedc0 2024-01-25T16:16:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 Processor -- start 2024-01-25T16:16:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 -- start start 2024-01-25T16:16:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 0x5562d5710f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5711450 0x5562d571e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d5720f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5680070 con 0x5562d5710b30 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d567d760 con 0x5562d5711450 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5679050 con 0x5562d571eb90 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89ea7fc640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 0x7f89e00a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 0x5562d5710f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f8b87640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d5720f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 0x5562d5710f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34588/0 (socket says 172.21.15.33:34588) 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f9388640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5711450 0x5562d571e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f9b89640 1 -- 172.21.15.33:0/1198294521 learned_addr learned my addr 172.21.15.33:0/1198294521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.133+0000 7f89f9b89640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 0x7f89e00a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89ea7fc640 1 -- 172.21.15.33:0/236278097 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89f0014070 con 0x7f89f40fedc0 2024-01-25T16:16:34.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 -- 172.21.15.33:0/1198294521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5711450 msgr2=0x5562d571e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5711450 0x5562d571e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 -- 172.21.15.33:0/1198294521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 msgr2=0x5562d5710f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 0x5562d5710f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 -- 172.21.15.33:0/1198294521 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d5670e70 con 0x5562d571eb90 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f8b87640 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d5720f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89e4000c00 tx=0x7f89e4005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b7265ae191a1e29 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89f9b89640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 0x7f89e00a39b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0099e0 tx=0x7f89ec009500 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89df7fe640 1 -- 172.21.15.33:0/1198294521 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40046c0 con 0x5562d571eb90 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89df7fe640 1 -- 172.21.15.33:0/1198294521 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e4004820 con 0x5562d571eb90 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.134+0000 7f89df7fe640 1 -- 172.21.15.33:0/1198294521 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4005220 con 0x5562d571eb90 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- 172.21.15.33:0/1198294521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 msgr2=0x5562d5720f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d5720f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f89e4000c00 tx=0x7f89e4005e40 comp rx=0 tx=0).stop 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- 172.21.15.33:0/1198294521 shutdown_connections 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d5720f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5711450 0x5562d571e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/1198294521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5710b30 0x5562d5710f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- 172.21.15.33:0/1198294521 >> 172.21.15.33:0/1198294521 conn(0x5562d570b9b0 msgr2=0x5562d570bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- 172.21.15.33:0/1198294521 shutdown_connections 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- 172.21.15.33:0/1198294521 wait complete. 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 -- start start 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.135+0000 7f89fe1b4740 1 Processor -- start 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d571c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d56321e0 con 0x5562d5711450 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5631e80 con 0x5562d5710b30 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d55f70c0 con 0x5562d571eb90 2024-01-25T16:16:34.144 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 10 client.0 _ll_register_callbacks cb 0x5562d561cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34604/0 (socket says 172.21.15.33:34604) 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 -- 172.21.15.33:0/624205170 learned_addr learned my addr 172.21.15.33:0/624205170 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f9388640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f9b89640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d571c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 -- 172.21.15.33:0/624205170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 msgr2=0x5562d571c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d571c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 -- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 msgr2=0x5562d571b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 -- 172.21.15.33:0/624205170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f410e3f0 con 0x5562d5711450 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89f8b87640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89e402c170 tx=0x7f89e402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab93d090bc76f874 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:34.145 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4005200 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89e4005360 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.136+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5562d5672040 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e40044e0 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 10 client.16101 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5562d56776d0 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5562d5672040 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f89e403c050 con 0x5562d5711450 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 10 client.16101 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 20 client.16101 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d572e5d0 0x5562d5730a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 4 client.16101 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5562d5671d60 con 0x5562d572e5d0 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89f9388640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d572e5d0 0x5562d5730a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.146 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89dcff9640 10 client.16101 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.137+0000 7f89f9388640 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d572e5d0 0x5562d5730a10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa0f857cb9cf5fc5 server_cookie=99db1aac97fe64b9 in_seq=0 out_seq=0 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89dcff9640 1 -- 172.21.15.33:0/624205170 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5562d5671d60 con 0x5562d572e5d0 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89dcff9640 10 client.16101 handle_command_reply: tid=0 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 client.16101 shutdown 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 msgr2=0x5562d571bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f89e402c170 tx=0x7f89e402bc30 comp rx=0 tx=0).stop 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 shutdown_connections 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d572e5d0 0x5562d5730a10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d571eb90 0x5562d571c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d5711450 0x5562d571bf30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89dcff9640 0 client.16101 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/624205170 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 >> 172.21.15.33:0/624205170 conn(0x5562d570b9b0 msgr2=0x5562d570d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.147 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 shutdown_connections 2024-01-25T16:16:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.138+0000 7f89fe1b4740 1 -- 172.21.15.33:0/624205170 wait complete. 2024-01-25T16:16:34.148 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.139+0000 7f89fe1b4740 20 client.16101 trim_cache size 0 max 16384 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 Processor -- start 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 -- start start 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d571c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d570b280 con 0x5562d571eb90 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d57359e0 con 0x5562d5710b30 2024-01-25T16:16:34.159 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5671d90 con 0x5562d5711450 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34606/0 (socket says 172.21.15.33:34606) 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 -- 172.21.15.33:0/2180084555 learned_addr learned my addr 172.21.15.33:0/2180084555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f8b87640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d571c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9388640 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 -- 172.21.15.33:0/2180084555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 msgr2=0x5562d571c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d571c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.160 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 -- 172.21.15.33:0/2180084555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 msgr2=0x5562d571e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.153+0000 7f89f9b89640 1 -- 172.21.15.33:0/2180084555 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562d5614060 con 0x5562d571eb90 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f89ec011fc0 tx=0x7f89ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e72055b85f2c4f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89deffd640 1 -- 172.21.15.33:0/2180084555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec025070 con 0x5562d571eb90 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89deffd640 1 -- 172.21.15.33:0/2180084555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ec011a50 con 0x5562d571eb90 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89deffd640 1 -- 172.21.15.33:0/2180084555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0046e0 con 0x5562d571eb90 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2180084555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 msgr2=0x5562d571a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f89ec011fc0 tx=0x7f89ec009b30 comp rx=0 tx=0).stop 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2180084555 shutdown_connections 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d571a9d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d571c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2180084555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d571e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2180084555 >> 172.21.15.33:0/2180084555 conn(0x5562d570b9b0 msgr2=0x5562d571a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2180084555 shutdown_connections 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.154+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2180084555 wait complete. 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 -- start start 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 Processor -- start 2024-01-25T16:16:34.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d5725da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d57241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5724ff0 con 0x5562d571eb90 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5784b00 con 0x5562d5710b30 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562d5784c40 con 0x5562d5711450 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89fe1b4740 10 client.0 _ll_register_callbacks cb 0x7f89f0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89f9388640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d5725da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89f9b89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34610/0 (socket says 172.21.15.33:34610) 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89f9b89640 1 -- 172.21.15.33:0/2275384999 learned_addr learned my addr 172.21.15.33:0/2275384999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.155+0000 7f89f8b87640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d57241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 -- 172.21.15.33:0/2275384999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 msgr2=0x5562d57241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d57241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 -- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 msgr2=0x5562d5725da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d5725da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 -- 172.21.15.33:0/2275384999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89ec019040 con 0x5562d571eb90 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89f9b89640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89ec003f30 tx=0x7f89ec005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c415d00fda0dc0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec025030 con 0x5562d571eb90 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f89ec004360 con 0x5562d571eb90 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89e4004820 con 0x5562d571eb90 2024-01-25T16:16:34.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ec004e50 con 0x5562d571eb90 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 10 client.16107 fetch_fsmap learned FSMap version 31 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5562d5639a80 con 0x5562d571eb90 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89ec03f080 con 0x5562d571eb90 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 31) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f89ec005780 con 0x5562d571eb90 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 10 client.16107 fetch_fsmap finished waiting for FSMap version 31 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 10 client.16107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 20 client.16107 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.156+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d57863d0 0x5562d57887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.157+0000 7f89fe1b4740 4 client.16107 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.157+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5562d5378600 con 0x5562d57863d0 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.157+0000 7f89f9388640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d57863d0 0x5562d57887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.157+0000 7f89dcff9640 10 client.16107 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:34.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.157+0000 7f89f9388640 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d57863d0 0x5562d57887f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=474fea2601ec70bb server_cookie=a8b8fa32ed832600 in_seq=0 out_seq=0 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89dcff9640 1 -- 172.21.15.33:0/2275384999 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5562d5378600 con 0x5562d57863d0 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89dcff9640 10 client.16107 handle_command_reply: tid=0 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 client.16107 shutdown 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 msgr2=0x5562d5724920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f89ec003f30 tx=0x7f89ec005bd0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 shutdown_connections 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5562d57863d0 0x5562d57887f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.300+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5562d571eb90 0x5562d5724920 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89dcff9640 0 client.16107 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5562d5711450 0x5562d57241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 1 --2- 172.21.15.33:0/2275384999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5562d5710b30 0x5562d5725da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 >> 172.21.15.33:0/2275384999 conn(0x5562d570b9b0 msgr2=0x5562d570ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 shutdown_connections 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 1 -- 172.21.15.33:0/2275384999 wait complete. 2024-01-25T16:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.301+0000 7f89fe1b4740 20 client.16107 trim_cache size 0 max 16384 2024-01-25T16:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 msgr2=0x7f89e00a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 0x7f89e00a39b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0099e0 tx=0x7f89ec009500 comp rx=0 tx=0).stop 2024-01-25T16:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 msgr2=0x7f89f4197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f89f000e9f0 tx=0x7f89f000edf0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 shutdown_connections 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f89e00a14f0 0x7f89e00a39b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f4106dd0 0x7f89f419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f89f40ff780 0x7f89f4197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 --2- 172.21.15.33:0/236278097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89f40fedc0 0x7f89f4197110 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 >> 172.21.15.33:0/236278097 conn(0x7f89f4075d80 msgr2=0x7f89f40fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 shutdown_connections 2024-01-25T16:16:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:34.302+0000 7f89fb5ec640 1 -- 172.21.15.33:0/236278097 wait complete. 2024-01-25T16:16:34.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 530365 7.73 MB/sec execute 331 sec latency 86.835 ms 2024-01-25T16:16:34.882 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:16:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:16:34.882 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:16:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:16:34.882 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:16:34 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/476207640 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560e612fd520 con 0x560e612d2c00 2024-01-25T16:16:34.882 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:16:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:16:35.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:35 smithi033 ceph-mon[31417]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-01-25T16:16:35.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:35 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:16:35.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:35 smithi152 ceph-mon[36015]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-01-25T16:16:35.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:35 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:16:35.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:35 smithi076 ceph-mon[35642]: pgmap v273: 97 pgs: 97 active+clean; 97 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 198 op/s 2024-01-25T16:16:35.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:35 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:16:35.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532549 7.74 MB/sec execute 332 sec latency 97.838 ms 2024-01-25T16:16:36.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:36 smithi033 ceph-mon[31417]: osdmap e98: 12 total, 12 up, 12 in 2024-01-25T16:16:36.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:36 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/1571602277,v1:172.21.15.152:6837/1571602277] up:boot 2024-01-25T16:16:36.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:36 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:36.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:36 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:16:36.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:36 smithi033 ceph-mon[31417]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:36 smithi152 ceph-mon[36015]: osdmap e98: 12 total, 12 up, 12 in 2024-01-25T16:16:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:36 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/1571602277,v1:172.21.15.152:6837/1571602277] up:boot 2024-01-25T16:16:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:36 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:36 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:16:36.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:36 smithi152 ceph-mon[36015]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:36.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:36 smithi076 ceph-mon[35642]: osdmap e98: 12 total, 12 up, 12 in 2024-01-25T16:16:36.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:36 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/1571602277,v1:172.21.15.152:6837/1571602277] up:boot 2024-01-25T16:16:36.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:36 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:16:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:36 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:16:36.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:36 smithi076 ceph-mon[35642]: pgmap v275: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:36.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 534417 7.75 MB/sec execute 333 sec latency 21.327 ms 2024-01-25T16:16:37.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 536776 7.77 MB/sec execute 334 sec latency 13.259 ms 2024-01-25T16:16:38.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:38 smithi033 ceph-mon[31417]: pgmap v276: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:38.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:38 smithi152 ceph-mon[36015]: pgmap v276: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:38.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:38 smithi076 ceph-mon[35642]: pgmap v276: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:38.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 538886 7.77 MB/sec execute 335 sec latency 19.765 ms 2024-01-25T16:16:39.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540903 7.78 MB/sec execute 336 sec latency 15.533 ms 2024-01-25T16:16:40.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 542969 7.80 MB/sec execute 337 sec latency 54.868 ms 2024-01-25T16:16:41.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:41 smithi033 ceph-mon[31417]: pgmap v277: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:41.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:41 smithi152 ceph-mon[36015]: pgmap v277: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:41.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:41 smithi076 ceph-mon[35642]: pgmap v277: 97 pgs: 97 active+clean; 101 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.5 MiB/s wr, 377 op/s 2024-01-25T16:16:41.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 544778 7.80 MB/sec execute 338 sec latency 17.211 ms 2024-01-25T16:16:42.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547283 7.83 MB/sec execute 339 sec latency 13.993 ms 2024-01-25T16:16:43.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:43 smithi033 ceph-mon[31417]: pgmap v278: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:43.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:43 smithi152 ceph-mon[36015]: pgmap v278: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:43.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:43 smithi076 ceph-mon[35642]: pgmap v278: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:43.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 549161 7.82 MB/sec execute 340 sec latency 19.538 ms 2024-01-25T16:16:44.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 551201 7.84 MB/sec execute 341 sec latency 13.960 ms 2024-01-25T16:16:45.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:45 smithi033 ceph-mon[31417]: pgmap v279: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:45.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:45 smithi152 ceph-mon[36015]: pgmap v279: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:45.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:45 smithi076 ceph-mon[35642]: pgmap v279: 97 pgs: 97 active+clean; 108 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 379 op/s 2024-01-25T16:16:45.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 553191 7.84 MB/sec execute 342 sec latency 46.321 ms 2024-01-25T16:16:46.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 555074 7.85 MB/sec execute 343 sec latency 16.596 ms 2024-01-25T16:16:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 -- 172.21.15.33:0/2975009361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106de0 msgr2=0x7f3eb41071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2975009361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106de0 0x7f3eb41071c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009900 tx=0x7f3ea802bcd0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 -- 172.21.15.33:0/2975009361 shutdown_connections 2024-01-25T16:16:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2975009361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068d10 0x7f3eb410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2975009361 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb4068370 0x7f3eb40687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2975009361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4106de0 0x7f3eb41071c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.394+0000 7f3ebae15640 1 -- 172.21.15.33:0/2975009361 >> 172.21.15.33:0/2975009361 conn(0x7f3eb4076a40 msgr2=0x7f3eb4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- 172.21.15.33:0/2975009361 shutdown_connections 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- 172.21.15.33:0/2975009361 wait complete. 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 Processor -- start 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- start start 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068370 0x7f3eb4197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb4106de0 0x7f3eb419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419bfc0 con 0x7f3eb4068d10 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419c130 con 0x7f3eb4068370 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3ebae15640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb419c2f0 con 0x7f3eb4106de0 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56018/0 (socket says 172.21.15.33:56018) 2024-01-25T16:16:47.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3eb3fff640 1 -- 172.21.15.33:0/2567513628 learned_addr learned my addr 172.21.15.33:0/2567513628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.395+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068370 0x7f3eb4197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 -- 172.21.15.33:0/2567513628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb4106de0 msgr2=0x7f3eb419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb4106de0 0x7f3eb419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 -- 172.21.15.33:0/2567513628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068370 msgr2=0x7f3eb4197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068370 0x7f3eb4197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 -- 172.21.15.33:0/2567513628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea8009560 con 0x7f3eb4068d10 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9f0 tx=0x7f3ea000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb1ffb640 1 -- 172.21.15.33:0/2567513628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0009720 con 0x7f3eb4068d10 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb1ffb640 1 -- 172.21.15.33:0/2567513628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ea000cd80 con 0x7f3eb4068d10 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb419c5d0 con 0x7f3eb4068d10 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3eb1ffb640 1 -- 172.21.15.33:0/2567513628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0010430 con 0x7f3eb4068d10 2024-01-25T16:16:47.400 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.396+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb41a3e50 con 0x7f3eb4068d10 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.398+0000 7f3ebd9d7740 1 Processor -- start 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.398+0000 7f3eb1ffb640 1 -- 172.21.15.33:0/2567513628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f3ea0005000 con 0x7f3eb4068d10 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 -- start start 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc7790d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77873070 con 0x55bc7790d780 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77870760 con 0x55bc779049b0 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc7786c050 con 0x55bc77904090 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb93b2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb93b2640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:48494/0 (socket says 172.21.15.33:48494) 2024-01-25T16:16:47.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb93b2640 1 -- 172.21.15.33:0/1301858567 learned_addr learned my addr 172.21.15.33:0/1301858567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc7790d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb1ffb640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 0x7f3ea40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 -- 172.21.15.33:0/1301858567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 msgr2=0x55bc77904470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 -- 172.21.15.33:0/1301858567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 msgr2=0x55bc7790d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc7790d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb3fff640 1 -- 172.21.15.33:0/1301858567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc77863e70 con 0x55bc7790d780 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 0x7f3ea40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.399+0000 7f3eb1ffb640 1 -- 172.21.15.33:0/2567513628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ea0014070 con 0x7f3eb4068d10 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790fb70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00caa10 tx=0x7f3ea0004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c026f3477bf101 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3e9effd640 1 -- 172.21.15.33:0/1301858567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0099e90 con 0x55bc7790d780 2024-01-25T16:16:47.408 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3e9effd640 1 -- 172.21.15.33:0/1301858567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ea0099c00 con 0x55bc7790d780 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3e9effd640 1 -- 172.21.15.33:0/1301858567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea009e210 con 0x55bc7790d780 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/1301858567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 msgr2=0x55bc7790fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 0x7f3ea40a39b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0049d0 tx=0x7f3eac004a40 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790fb70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00caa10 tx=0x7f3ea0004a20 comp rx=0 tx=0).stop 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/1301858567 shutdown_connections 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790fb70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc7790d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/1301858567 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc77904090 0x55bc77904470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.400+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/1301858567 >> 172.21.15.33:0/1301858567 conn(0x55bc778fe9b0 msgr2=0x55bc778fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/1301858567 shutdown_connections 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/1301858567 wait complete. 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- start start 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 Processor -- start 2024-01-25T16:16:47.409 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 0x55bc7790a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc778251e0 con 0x55bc7790d780 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77824e80 con 0x55bc77904090 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.401+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc777ea0c0 con 0x55bc779049b0 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3ebd9d7740 10 client.0 _ll_register_callbacks cb 0x55bc7780fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56036/0 (socket says 172.21.15.33:56036) 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb8bb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 0x55bc7790a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 -- 172.21.15.33:0/937950676 learned_addr learned my addr 172.21.15.33:0/937950676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 -- 172.21.15.33:0/937950676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 msgr2=0x55bc7790ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 -- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 msgr2=0x55bc7790a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 0x55bc7790a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 -- 172.21.15.33:0/937950676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb41a3e50 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 0x55bc7790a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3eac011b10 tx=0x7f3eac009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b927c4f6df4c1f83 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3e93fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac012ea0 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3eac01d450 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.402+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc77865040 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eac01d610 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 10 client.16125 fetch_fsmap learned FSMap version 32 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc7786a6d0 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc77865040 con 0x55bc7790d780 2024-01-25T16:16:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3eac01e970 con 0x55bc7790d780 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 10 client.16125 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 10 client.16125 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 20 client.16125 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc7791d540 0x55bc7791f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 4 client.16125 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc77864d60 con 0x55bc7791d540 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.403+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc7791d540 0x55bc7791f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.404+0000 7f3e93fff640 10 client.16125 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.404+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc7791d540 0x55bc7791f960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc7a3a3387207f4 server_cookie=c9b6e48ee5d69dbc in_seq=0 out_seq=0 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.404+0000 7f3e93fff640 1 -- 172.21.15.33:0/937950676 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc77864d60 con 0x55bc7791d540 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.404+0000 7f3e93fff640 10 client.16125 handle_command_reply: tid=0 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.404+0000 7f3ebd9d7740 1 client.16125 shutdown 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 msgr2=0x55bc7790b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3eac011b10 tx=0x7f3eac009500 comp rx=0 tx=0).stop 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 shutdown_connections 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc7791d540 0x55bc7791f960 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc7790d780 0x55bc7790b0d0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/937950676 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc77904090 0x55bc7790a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3e93fff640 0 client.16125 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 >> 172.21.15.33:0/937950676 conn(0x55bc778fe9b0 msgr2=0x55bc77900700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 shutdown_connections 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/937950676 wait complete. 2024-01-25T16:16:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.405+0000 7f3ebd9d7740 20 client.16125 trim_cache size 0 max 16384 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 Processor -- start 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 -- start start 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc7790d780 0x55bc77908f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc778ff7b0 con 0x55bc77904090 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc7790d600 con 0x55bc7790d780 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.418+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc7791bc70 con 0x55bc779049b0 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.421 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56038/0 (socket says 172.21.15.33:56038) 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/495679129 learned_addr learned my addr 172.21.15.33:0/495679129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc7790d780 0x55bc77908f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/495679129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 msgr2=0x55bc7790aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/495679129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc7790d780 msgr2=0x55bc77908f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc7790d780 0x55bc77908f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/495679129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc77807060 con 0x55bc77904090 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009230 tx=0x7f3ea802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109a96db67e240a2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3e9e7fc640 1 -- 172.21.15.33:0/495679129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8046070 con 0x55bc77904090 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3e9e7fc640 1 -- 172.21.15.33:0/495679129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ea804b070 con 0x55bc77904090 2024-01-25T16:16:47.422 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.419+0000 7f3e9e7fc640 1 -- 172.21.15.33:0/495679129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea802cc40 con 0x55bc77904090 2024-01-25T16:16:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/495679129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 msgr2=0x55bc7790cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009230 tx=0x7f3ea802e040 comp rx=0 tx=0).stop 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/495679129 shutdown_connections 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc7790d780 0x55bc77908f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc779049b0 0x55bc7790aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/495679129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc7790cd80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/495679129 >> 172.21.15.33:0/495679129 conn(0x55bc778fe9b0 msgr2=0x55bc779242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/495679129 shutdown_connections 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/495679129 wait complete. 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 -- start start 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.420+0000 7f3ebd9d7740 1 Processor -- start 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc779128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc7790d780 0x55bc77912e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77914cf0 con 0x55bc77904090 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77913360 con 0x55bc779049b0 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc77913520 con 0x55bc7790d780 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3ebd9d7740 10 client.0 _ll_register_callbacks cb 0x7f3ea00d0af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56048/0 (socket says 172.21.15.33:56048) 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/2995576339 learned_addr learned my addr 172.21.15.33:0/2995576339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc779128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb93b2640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc7790d780 0x55bc77912e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/2995576339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc7790d780 msgr2=0x55bc77912e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc7790d780 0x55bc77912e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 msgr2=0x55bc779128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc779128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.421+0000 7f3eb8bb1640 1 -- 172.21.15.33:0/2995576339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb410e3c0 con 0x55bc77904090 2024-01-25T16:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3eb8bb1640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005df0 tx=0x7f3ea8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a07e87ed242f089 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8046040 con 0x55bc77904090 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ea802cae0 con 0x55bc77904090 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ea0099c00 con 0x55bc77904090 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ea802cca0 con 0x55bc77904090 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 10 client.16131 fetch_fsmap learned FSMap version 32 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55bc7782ca80 con 0x55bc77904090 2024-01-25T16:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ea0099c00 con 0x55bc77904090 2024-01-25T16:16:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f3ea803c040 con 0x55bc77904090 2024-01-25T16:16:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 10 client.16131 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 20 client.16131 populate_metadata read hostname 'smithi033' 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc77974c60 0x55bc77977080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 4 client.16131 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc7756b600 con 0x55bc77974c60 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.422+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc77974c60 0x55bc77977080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.423+0000 7f3e93fff640 10 client.16131 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.423+0000 7f3eb3fff640 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc77974c60 0x55bc77977080 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5495d6ef204dcb72 server_cookie=1bd1d0c1e879fc44 in_seq=0 out_seq=0 2024-01-25T16:16:47.678 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:47 smithi076 ceph-mon[35642]: pgmap v280: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-01-25T16:16:47.678 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:47 smithi152 ceph-mon[36015]: pgmap v280: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-01-25T16:16:47.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:47 smithi033 ceph-mon[31417]: pgmap v280: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 457 op/s 2024-01-25T16:16:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3e93fff640 1 -- 172.21.15.33:0/2995576339 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc7756b600 con 0x55bc77974c60 2024-01-25T16:16:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3e93fff640 10 client.16131 handle_command_reply: tid=0 2024-01-25T16:16:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 client.16131 shutdown 2024-01-25T16:16:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 msgr2=0x55bc77914440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8005df0 tx=0x7f3ea8009260 comp rx=0 tx=0).stop 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 shutdown_connections 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55bc77974c60 0x55bc77977080 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55bc7790d780 0x55bc77912e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bc779049b0 0x55bc779128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 --2- 172.21.15.33:0/2995576339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc77904090 0x55bc77914440 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 >> 172.21.15.33:0/2995576339 conn(0x55bc778fe9b0 msgr2=0x55bc77908ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3e93fff640 0 client.16131 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 shutdown_connections 2024-01-25T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 1 -- 172.21.15.33:0/2995576339 wait complete. 2024-01-25T16:16:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.577+0000 7f3ebd9d7740 20 client.16131 trim_cache size 0 max 16384 2024-01-25T16:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:16:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 msgr2=0x7f3ea40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 0x7f3ea40a39b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0049d0 tx=0x7f3eac004a40 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 msgr2=0x7f3eb4197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9f0 tx=0x7f3ea000edf0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 shutdown_connections 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f3ea40a14f0 0x7f3ea40a39b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3eb4106de0 0x7f3eb419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3eb4068d10 0x7f3eb4197690 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 --2- 172.21.15.33:0/2567513628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3eb4068370 0x7f3eb4197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:16:47.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 >> 172.21.15.33:0/2567513628 conn(0x7f3eb4076a40 msgr2=0x7f3eb40fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:16:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.578+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 shutdown_connections 2024-01-25T16:16:47.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:16:47.579+0000 7f3ebae15640 1 -- 172.21.15.33:0/2567513628 wait complete. 2024-01-25T16:16:47.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 556651 7.85 MB/sec execute 344 sec latency 134.615 ms 2024-01-25T16:16:48.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 558764 7.86 MB/sec execute 345 sec latency 14.953 ms 2024-01-25T16:16:49.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:49 smithi033 ceph-mon[31417]: pgmap v281: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:49.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:49 smithi152 ceph-mon[36015]: pgmap v281: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:49.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:49 smithi076 ceph-mon[35642]: pgmap v281: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:49.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 561368 7.89 MB/sec execute 346 sec latency 16.988 ms 2024-01-25T16:16:50.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 562903 7.88 MB/sec execute 347 sec latency 60.513 ms 2024-01-25T16:16:51.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:51 smithi033 ceph-mon[31417]: pgmap v282: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:51.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:51 smithi152 ceph-mon[36015]: pgmap v282: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:51.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:51 smithi076 ceph-mon[35642]: pgmap v282: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 205 op/s 2024-01-25T16:16:51.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 565187 7.90 MB/sec execute 348 sec latency 28.939 ms 2024-01-25T16:16:52.781 INFO:tasks.workunit.client.0.smithi033.stdout: 1 567054 7.89 MB/sec execute 349 sec latency 94.093 ms 2024-01-25T16:16:53.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:53 smithi033 ceph-mon[31417]: pgmap v283: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-01-25T16:16:53.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:53 smithi152 ceph-mon[36015]: pgmap v283: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-01-25T16:16:53.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:53 smithi076 ceph-mon[35642]: pgmap v283: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 385 op/s 2024-01-25T16:16:53.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568946 7.91 MB/sec execute 350 sec latency 14.340 ms 2024-01-25T16:16:54.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571347 7.92 MB/sec execute 351 sec latency 18.950 ms 2024-01-25T16:16:55.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:55 smithi033 ceph-mon[31417]: pgmap v284: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 282 op/s 2024-01-25T16:16:55.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:55 smithi152 ceph-mon[36015]: pgmap v284: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 282 op/s 2024-01-25T16:16:55.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:55 smithi076 ceph-mon[35642]: pgmap v284: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 282 op/s 2024-01-25T16:16:55.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 573139 7.92 MB/sec execute 352 sec latency 40.541 ms 2024-01-25T16:16:56.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 575503 7.95 MB/sec execute 353 sec latency 43.393 ms 2024-01-25T16:16:57.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:57 smithi033 ceph-mon[31417]: pgmap v285: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 374 op/s 2024-01-25T16:16:57.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:57 smithi152 ceph-mon[36015]: pgmap v285: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 374 op/s 2024-01-25T16:16:57.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:57 smithi076 ceph-mon[35642]: pgmap v285: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 374 op/s 2024-01-25T16:16:57.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 577025 7.94 MB/sec execute 354 sec latency 44.392 ms 2024-01-25T16:16:58.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 579274 7.96 MB/sec execute 355 sec latency 62.192 ms 2024-01-25T16:16:59.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:16:59 smithi033 ceph-mon[31417]: pgmap v286: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:16:59.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:16:59 smithi152 ceph-mon[36015]: pgmap v286: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:16:59.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:16:59 smithi076 ceph-mon[35642]: pgmap v286: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:16:59.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 581311 7.96 MB/sec execute 356 sec latency 21.687 ms 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 -- 172.21.15.33:0/1631709611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc073120 msgr2=0x7f91fc073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 --2- 172.21.15.33:0/1631709611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc073120 0x7f91fc073500 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f91f8009900 tx=0x7f91f802bcd0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 -- 172.21.15.33:0/1631709611 shutdown_connections 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 --2- 172.21.15.33:0/1631709611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc10d4e0 0x7f91fc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 --2- 172.21.15.33:0/1631709611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073a40 0x7f91fc10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 --2- 172.21.15.33:0/1631709611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc073120 0x7f91fc073500 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 -- 172.21.15.33:0/1631709611 >> 172.21.15.33:0/1631709611 conn(0x7f91fc0fc770 msgr2=0x7f91fc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 -- 172.21.15.33:0/1631709611 shutdown_connections 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.671+0000 7f9203d05640 1 -- 172.21.15.33:0/1631709611 wait complete. 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 Processor -- start 2024-01-25T16:17:00.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 -- start start 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 0x7f91fc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 0x7f91fc1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc19e270 con 0x7f91fc10d4e0 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc19e3e0 con 0x7f91fc073120 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9203d05640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91fc19e5a0 con 0x7f91fc073a40 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53484/0 (socket says 172.21.15.33:53484) 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f92022a2640 1 -- 172.21.15.33:0/2178208777 learned_addr learned my addr 172.21.15.33:0/2178208777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f9201aa1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 0x7f91fc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.672+0000 7f92012a0640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 0x7f91fc1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 -- 172.21.15.33:0/2178208777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 msgr2=0x7f91fc1998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 0x7f91fc1998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 -- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 msgr2=0x7f91fc19dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 -- 172.21.15.33:0/2178208777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f8009560 con 0x7f91fc073120 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f92022a2640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f92012a0640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 0x7f91fc1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 0x7f91fc1993b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91f8036040 tx=0x7f91f80045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f91f2ffd640 1 -- 172.21.15.33:0/2178208777 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f802ca30 con 0x7f91fc073120 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f91f2ffd640 1 -- 172.21.15.33:0/2178208777 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f80054c0 con 0x7f91fc073120 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91fc19e820 con 0x7f91fc073120 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f91f2ffd640 1 -- 172.21.15.33:0/2178208777 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8003a80 con 0x7f91fc073120 2024-01-25T16:17:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.673+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc1a6100 con 0x7f91fc073120 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.675+0000 7f91f2ffd640 1 -- 172.21.15.33:0/2178208777 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f91f802cb90 con 0x7f91fc073120 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 Processor -- start 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 -- start start 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f91f2ffd640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 0x7f91e80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92022a2640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 0x7f91e80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90ac780 0x5599b90aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f91f2ffd640 1 -- 172.21.15.33:0/2178208777 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91f80eef20 con 0x7f91fc073120 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b9012070 con 0x5599b90a3090 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.676+0000 7f92069a5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b900f760 con 0x5599b90ac780 2024-01-25T16:17:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f92069a5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b900b050 con 0x5599b90a39b0 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53494/0 (socket says 172.21.15.33:53494) 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 -- 172.21.15.33:0/385713016 learned_addr learned my addr 172.21.15.33:0/385713016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f92022a2640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90ac780 0x5599b90aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f92012a0640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 -- 172.21.15.33:0/385713016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 msgr2=0x5599b90ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 -- 172.21.15.33:0/385713016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90ac780 msgr2=0x5599b90aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90ac780 0x5599b90aeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 -- 172.21.15.33:0/385713016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b9002e70 con 0x5599b90a3090 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f92012a0640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f92022a2640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 0x7f91e80a39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009fd0 tx=0x7f91f400ff30 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f9201aa1640 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f91f80c3c80 tx=0x7f91f80bb840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e912646dc8c2900 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f91e7fff640 1 -- 172.21.15.33:0/385713016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80bff70 con 0x5599b90a3090 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f91e7fff640 1 -- 172.21.15.33:0/385713016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f80c41b0 con 0x5599b90a3090 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.677+0000 7f91e7fff640 1 -- 172.21.15.33:0/385713016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80bc2e0 con 0x5599b90a3090 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- 172.21.15.33:0/385713016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 msgr2=0x5599b90a3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f91f80c3c80 tx=0x7f91f80bb840 comp rx=0 tx=0).stop 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- 172.21.15.33:0/385713016 shutdown_connections 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90ac780 0x5599b90aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 --2- 172.21.15.33:0/385713016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a3090 0x5599b90a3470 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- 172.21.15.33:0/385713016 >> 172.21.15.33:0/385713016 conn(0x5599b909d9b0 msgr2=0x5599b909dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- 172.21.15.33:0/385713016 shutdown_connections 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- 172.21.15.33:0/385713016 wait complete. 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.678+0000 7f92069a5740 1 -- start start 2024-01-25T16:17:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 Processor -- start 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90ac780 0x5599b90aa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b8fc41e0 con 0x5599b90a39b0 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b8fc3e80 con 0x5599b90a3090 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b8f890c0 con 0x5599b90ac780 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92069a5740 10 client.0 _ll_register_callbacks cb 0x5599b8faeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53506/0 (socket says 172.21.15.33:53506) 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f9201aa1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 -- 172.21.15.33:0/2043269269 learned_addr learned my addr 172.21.15.33:0/2043269269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92022a2640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90ac780 0x5599b90aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.687 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 -- 172.21.15.33:0/2043269269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90ac780 msgr2=0x5599b90aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90ac780 0x5599b90aa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 -- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 msgr2=0x5599b90a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f92012a0640 1 -- 172.21.15.33:0/2043269269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91fc1a6100 con 0x5599b90a39b0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.679+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92012a0640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f91ec013d30 tx=0x7f91ec0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e758dc08584b71c0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ec009ea0 con 0x5599b90a39b0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91ec005220 con 0x5599b90a39b0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599b9004040 con 0x5599b90a39b0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91ec0053e0 con 0x5599b90a39b0 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 10 client.16143 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:00.688 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5599b90096d0 con 0x5599b90a39b0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599b9004040 con 0x5599b90a39b0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f91ec01b730 con 0x5599b90a39b0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 10 client.16143 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 20 client.16143 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b90bc3c0 0x5599b90be7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 4 client.16143 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.680+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599b9003d60 con 0x5599b90bc3c0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.681+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b90bc3c0 0x5599b90be7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.681+0000 7f91e57fa640 10 client.16143 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.681+0000 7f9201aa1640 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b90bc3c0 0x5599b90be7e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f625259c2923aa3 server_cookie=cd2b6af3864ae7d4 in_seq=0 out_seq=0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f91e57fa640 1 -- 172.21.15.33:0/2043269269 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5599b9003d60 con 0x5599b90bc3c0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f91e57fa640 10 client.16143 handle_command_reply: tid=0 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 client.16143 shutdown 2024-01-25T16:17:00.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 msgr2=0x5599b90a9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f91ec013d30 tx=0x7f91ec0117b0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 shutdown_connections 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b90bc3c0 0x5599b90be7e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90ac780 0x5599b90aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90a39b0 0x5599b90a9c00 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 --2- 172.21.15.33:0/2043269269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f91e57fa640 0 client.16143 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 >> 172.21.15.33:0/2043269269 conn(0x5599b909d9b0 msgr2=0x5599b909f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 shutdown_connections 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 1 -- 172.21.15.33:0/2043269269 wait complete. 2024-01-25T16:17:00.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.682+0000 7f92069a5740 20 client.16143 trim_cache size 0 max 16384 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 Processor -- start 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 -- start start 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b909e9b0 con 0x5599b90ac780 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b90baa90 con 0x5599b90a3090 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92069a5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b909d810 con 0x5599b90a39b0 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53522/0 (socket says 172.21.15.33:53522) 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f92022a2640 1 -- 172.21.15.33:0/96030677 learned_addr learned my addr 172.21.15.33:0/96030677 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.695+0000 7f9201aa1640 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92012a0640 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 -- 172.21.15.33:0/96030677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 msgr2=0x5599b90a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 -- 172.21.15.33:0/96030677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 msgr2=0x5599b90abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 -- 172.21.15.33:0/96030677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599b8fa6060 con 0x5599b90ac780 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f92022a2640 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f91f4012300 tx=0x7f91f401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca06d68be5e757f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f91e77fe640 1 -- 172.21.15.33:0/96030677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4010e80 con 0x5599b90ac780 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f91e77fe640 1 -- 172.21.15.33:0/96030677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f4024070 con 0x5599b90ac780 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.696+0000 7f91e77fe640 1 -- 172.21.15.33:0/96030677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4028700 con 0x5599b90ac780 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- 172.21.15.33:0/96030677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 msgr2=0x5599b90a86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f91f4012300 tx=0x7f91f401f000 comp rx=0 tx=0).stop 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- 172.21.15.33:0/96030677 shutdown_connections 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90a86a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a39b0 0x5599b90a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 --2- 172.21.15.33:0/96030677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a3090 0x5599b90abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- 172.21.15.33:0/96030677 >> 172.21.15.33:0/96030677 conn(0x5599b909d9b0 msgr2=0x5599b90a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- 172.21.15.33:0/96030677 shutdown_connections 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- 172.21.15.33:0/96030677 wait complete. 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 -- start start 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.697+0000 7f92069a5740 1 Processor -- start 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a3090 0x5599b90b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a39b0 0x5599b90b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.704 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b90b25b0 con 0x5599b90ac780 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b90b2720 con 0x5599b90a39b0 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599b91128d0 con 0x5599b90a3090 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 10 client.0 _ll_register_callbacks cb 0x7f91f802ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53534/0 (socket says 172.21.15.33:53534) 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 -- 172.21.15.33:0/121800248 learned_addr learned my addr 172.21.15.33:0/121800248 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92012a0640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a39b0 0x5599b90b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f9201aa1640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a3090 0x5599b90b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 -- 172.21.15.33:0/121800248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a3090 msgr2=0x5599b90b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a3090 0x5599b90b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 -- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a39b0 msgr2=0x5599b90b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a39b0 0x5599b90b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.705 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 -- 172.21.15.33:0/121800248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f4012070 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92022a2640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91f400f6b0 tx=0x7f91f4009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d6143044376625 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f91e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f40285e0 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f91f4004a40 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91f80c41b0 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91f4028740 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.698+0000 7f92069a5740 10 client.16149 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5599b8fcba80 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f403e020 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f91f4020770 con 0x5599b90ac780 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 10 client.16149 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 20 client.16149 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:00.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b9114020 0x5599b9116440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 4 client.16149 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:17:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5599b8d0a600 con 0x5599b9114020 2024-01-25T16:17:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f9201aa1640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b9114020 0x5599b9116440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f91e57fa640 10 client.16149 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:00.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.699+0000 7f9201aa1640 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b9114020 0x5599b9116440 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706d22b1d84ed905 server_cookie=18571c009c82b443 in_seq=0 out_seq=0 2024-01-25T16:17:00.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582750 7.97 MB/sec execute 357 sec latency 44.376 ms 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f91e57fa640 1 -- 172.21.15.33:0/121800248 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5599b8d0a600 con 0x5599b9114020 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f91e57fa640 10 client.16149 handle_command_reply: tid=0 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 client.16149 shutdown 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 msgr2=0x5599b90b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f91f400f6b0 tx=0x7f91f4009ec0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 shutdown_connections 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5599b9114020 0x5599b9116440 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599b90ac780 0x5599b90b1ed0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5599b90a39b0 0x5599b90b1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 --2- 172.21.15.33:0/121800248 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5599b90a3090 0x5599b90b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f91e57fa640 0 client.16149 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 >> 172.21.15.33:0/121800248 conn(0x5599b909d9b0 msgr2=0x5599b909edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 shutdown_connections 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 1 -- 172.21.15.33:0/121800248 wait complete. 2024-01-25T16:17:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f92069a5740 20 client.16149 trim_cache size 0 max 16384 2024-01-25T16:17:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:17:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:17:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:17:00.870 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:17:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 msgr2=0x7f91e80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.866+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 0x7f91e80a39b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009fd0 tx=0x7f91f400ff30 comp rx=0 tx=0).stop 2024-01-25T16:17:00.871 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 msgr2=0x7f91fc1993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 0x7f91fc1993b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91f8036040 tx=0x7f91f80045e0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 shutdown_connections 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f91e80a14f0 0x7f91e80a39b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91fc10d4e0 0x7f91fc19dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f91fc073a40 0x7f91fc1998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 --2- 172.21.15.33:0/2178208777 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91fc073120 0x7f91fc1993b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 >> 172.21.15.33:0/2178208777 conn(0x7f91fc0fc770 msgr2=0x7f91fc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 shutdown_connections 2024-01-25T16:17:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:00.867+0000 7f9203d05640 1 -- 172.21.15.33:0/2178208777 wait complete. 2024-01-25T16:17:01.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:01 smithi033 ceph-mon[31417]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:17:01.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:01 smithi152 ceph-mon[36015]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:17:01.642 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:01 smithi076 ceph-mon[35642]: pgmap v287: 97 pgs: 97 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.1 MiB/s wr, 271 op/s 2024-01-25T16:17:01.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 584168 7.96 MB/sec execute 358 sec latency 113.944 ms 2024-01-25T16:17:02.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:02 smithi033 ceph-mon[31417]: pgmap v288: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-01-25T16:17:02.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:02 smithi033 ceph-mon[31417]: osdmap e99: 12 total, 12 up, 12 in 2024-01-25T16:17:02.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:02 smithi152 ceph-mon[36015]: pgmap v288: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-01-25T16:17:02.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:02 smithi152 ceph-mon[36015]: osdmap e99: 12 total, 12 up, 12 in 2024-01-25T16:17:02.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:02 smithi076 ceph-mon[35642]: pgmap v288: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 4.9 MiB/s wr, 480 op/s 2024-01-25T16:17:02.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:02 smithi076 ceph-mon[35642]: osdmap e99: 12 total, 12 up, 12 in 2024-01-25T16:17:02.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 585127 7.95 MB/sec execute 359 sec latency 108.787 ms 2024-01-25T16:17:03.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:03 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:17:03.633 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:03 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:17:03.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:03 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:17:03.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586521 7.95 MB/sec execute 360 sec latency 25.506 ms 2024-01-25T16:17:04.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:04 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:17:04.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:04 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:17:04.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:04 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:17:04.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:04 smithi033 ceph-mon[31417]: pgmap v290: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-01-25T16:17:04.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:04 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:17:04.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:04 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:17:04.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:04 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:17:04.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:04 smithi152 ceph-mon[36015]: pgmap v290: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-01-25T16:17:04.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:04 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:17:04.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:04 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:17:04.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:04 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:17:04.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:04 smithi076 ceph-mon[35642]: pgmap v290: 97 pgs: 97 active+clean; 112 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 361 op/s 2024-01-25T16:17:04.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 588410 7.95 MB/sec execute 361 sec latency 107.308 ms 2024-01-25T16:17:05.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.96 MB/sec execute 362 sec latency 416.037 ms 2024-01-25T16:17:06.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.94 MB/sec execute 363 sec latency 1416.077 ms 2024-01-25T16:17:07.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:07 smithi033 ceph-mon[31417]: pgmap v291: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:07.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:07 smithi152 ceph-mon[36015]: pgmap v291: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:07.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:07 smithi076 ceph-mon[35642]: pgmap v291: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:07.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.92 MB/sec execute 364 sec latency 2416.187 ms 2024-01-25T16:17:08.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.89 MB/sec execute 365 sec latency 3416.249 ms 2024-01-25T16:17:09.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:09 smithi033 ceph-mon[31417]: pgmap v292: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:09.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:09 smithi152 ceph-mon[36015]: pgmap v292: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:09.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:09 smithi076 ceph-mon[35642]: pgmap v292: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:09.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.87 MB/sec execute 366 sec latency 4416.317 ms 2024-01-25T16:17:10.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.85 MB/sec execute 367 sec latency 5416.468 ms 2024-01-25T16:17:11.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:11 smithi033 ceph-mon[31417]: pgmap v293: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:11.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:11 smithi033 ceph-mon[31417]: osdmap e100: 12 total, 12 up, 12 in 2024-01-25T16:17:11.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:11 smithi152 ceph-mon[36015]: pgmap v293: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:11.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:11 smithi152 ceph-mon[36015]: osdmap e100: 12 total, 12 up, 12 in 2024-01-25T16:17:11.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:11 smithi076 ceph-mon[35642]: pgmap v293: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.8 MiB/s wr, 354 op/s 2024-01-25T16:17:11.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:11 smithi076 ceph-mon[35642]: osdmap e100: 12 total, 12 up, 12 in 2024-01-25T16:17:11.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.83 MB/sec execute 368 sec latency 6416.515 ms 2024-01-25T16:17:12.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.81 MB/sec execute 369 sec latency 7416.618 ms 2024-01-25T16:17:13.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:13 smithi033 ceph-mon[31417]: pgmap v295: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:13.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:13 smithi152 ceph-mon[36015]: pgmap v295: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:13.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:13 smithi076 ceph-mon[35642]: pgmap v295: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:13.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589768 7.79 MB/sec execute 370 sec latency 8416.663 ms 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 -- 172.21.15.33:0/872754271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81018e0 msgr2=0x7f5bb8105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 --2- 172.21.15.33:0/872754271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81018e0 0x7f5bb8105920 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009930 tx=0x7f5bac02bc30 comp rx=0 tx=0).stop 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 -- 172.21.15.33:0/872754271 shutdown_connections 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 --2- 172.21.15.33:0/872754271 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8106000 0x7f5bb810dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 --2- 172.21.15.33:0/872754271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb81018e0 0x7f5bb8105920 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 --2- 172.21.15.33:0/872754271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8100f30 0x7f5bb8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 -- 172.21.15.33:0/872754271 >> 172.21.15.33:0/872754271 conn(0x7f5bb80fc7d0 msgr2=0x7f5bb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:13.967 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.962+0000 7f5bbd832640 1 -- 172.21.15.33:0/872754271 shutdown_connections 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 -- 172.21.15.33:0/872754271 wait complete. 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 Processor -- start 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 -- start start 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 0x7f5bb8073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.963+0000 7f5bbd832640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 0x7f5bb81a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbd832640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81a59a0 con 0x7f5bb8100f30 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbd832640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81a5b10 con 0x7f5bb8106000 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbd832640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb81a5cd0 con 0x7f5bb8073a70 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58074/0 (socket says 172.21.15.33:58074) 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/768841532 learned_addr learned my addr 172.21.15.33:0/768841532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:13.968 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 0x7f5bb8073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 0x7f5bb81a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/768841532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 msgr2=0x7f5bb81a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 0x7f5bb81a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/768841532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 msgr2=0x7f5bb8073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 0x7f5bb8073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/768841532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009560 con 0x7f5bb8100f30 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 0x7f5bb8073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 0x7f5bb81a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d950 tx=0x7f5ba800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbc830640 1 -- 172.21.15.33:0/768841532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8009c30 con 0x7f5bb8100f30 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bb81a5fb0 con 0x7f5bb8100f30 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbc830640 1 -- 172.21.15.33:0/768841532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ba8009d90 con 0x7f5bb8100f30 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbc830640 1 -- 172.21.15.33:0/768841532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba800f740 con 0x7f5bb8100f30 2024-01-25T16:17:13.969 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.964+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb81a63e0 con 0x7f5bb8100f30 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.967+0000 7f5bbc830640 1 -- 172.21.15.33:0/768841532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5ba8010a20 con 0x7f5bb8100f30 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.967+0000 7f5bc03f0740 1 Processor -- start 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.967+0000 7f5bc03f0740 1 -- start start 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc35090 0x564eddc35470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 0x564eddc3e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddba4070 con 0x564eddc3e780 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddba1760 con 0x564eddc35090 2024-01-25T16:17:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddb9d050 con 0x564eddc359b0 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58076/0 (socket says 172.21.15.33:58076) 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 -- 172.21.15.33:0/1689786193 learned_addr learned my addr 172.21.15.33:0/1689786193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bbc830640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 0x7f5ba00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 0x564eddc3e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc35090 0x564eddc35470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 -- 172.21.15.33:0/1689786193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 msgr2=0x564eddc3e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 0x564eddc3e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 0x7f5ba00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 -- 172.21.15.33:0/1689786193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc35090 msgr2=0x564eddc35470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc35090 0x564eddc35470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 -- 172.21.15.33:0/1689786193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eddb94e70 con 0x564eddc3e780 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 0x564eddc3e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:17:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bbc830640 1 -- 172.21.15.33:0/768841532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ba80ccc30 con 0x7f5bb8100f30 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.968+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bac000c00 tx=0x7f5bac004e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9922e9c821bb9615 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 0x7f5ba00a39e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005c40 tx=0x7f5bb0009200 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5b9dffb640 1 -- 172.21.15.33:0/1689786193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0049c0 con 0x564eddc3e780 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5b9dffb640 1 -- 172.21.15.33:0/1689786193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac004040 con 0x564eddc3e780 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5b9dffb640 1 -- 172.21.15.33:0/1689786193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac0053a0 con 0x564eddc3e780 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 -- 172.21.15.33:0/1689786193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 msgr2=0x564eddc40b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5bac000c00 tx=0x7f5bac004e50 comp rx=0 tx=0).stop 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 -- 172.21.15.33:0/1689786193 shutdown_connections 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc40b70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc359b0 0x564eddc3e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/1689786193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc35090 0x564eddc35470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 -- 172.21.15.33:0/1689786193 >> 172.21.15.33:0/1689786193 conn(0x564eddc2f9b0 msgr2=0x564eddc2fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 -- 172.21.15.33:0/1689786193 shutdown_connections 2024-01-25T16:17:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.969+0000 7f5bc03f0740 1 -- 172.21.15.33:0/1689786193 wait complete. 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 -- start start 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 Processor -- start 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc3b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddb561e0 con 0x564eddc3e780 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddb55e80 con 0x564eddc359b0 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddb1b0c0 con 0x564eddc35090 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bc03f0740 10 client.0 _ll_register_callbacks cb 0x564eddb40ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.970+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58088/0 (socket says 172.21.15.33:58088) 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 -- 172.21.15.33:0/3576061023 learned_addr learned my addr 172.21.15.33:0/3576061023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc3b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 -- 172.21.15.33:0/3576061023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 msgr2=0x564eddc3b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc3b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 -- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 msgr2=0x564eddc3bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 -- 172.21.15.33:0/3576061023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb810e3e0 con 0x564eddc3e780 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000f970 tx=0x7f5bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74c33545a4e91ab server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5b96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0004d20 con 0x564eddc3e780 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5bb0011970 con 0x564eddc3e780 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eddb96040 con 0x564eddc3e780 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb0004e80 con 0x564eddc3e780 2024-01-25T16:17:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bc03f0740 10 client.16161 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.971+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564eddb9b6d0 con 0x564eddc3e780 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564eddb96040 con 0x564eddc3e780 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5bb0005770 con 0x564eddc3e780 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 10 client.16161 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 20 client.16161 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddc4e540 0x564eddc50960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 4 client.16161 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eddb95d60 con 0x564eddc4e540 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.972+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddc4e540 0x564eddc50960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.973+0000 7f5b96ffd640 10 client.16161 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.973+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddc4e540 0x564eddc50960 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34383e366018981e server_cookie=9fc2ab3d4be867b2 in_seq=0 out_seq=0 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.973+0000 7f5b96ffd640 1 -- 172.21.15.33:0/3576061023 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564eddb95d60 con 0x564eddc4e540 2024-01-25T16:17:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.973+0000 7f5b96ffd640 10 client.16161 handle_command_reply: tid=0 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.973+0000 7f5bc03f0740 1 client.16161 shutdown 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 msgr2=0x564eddc3c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000f970 tx=0x7f5bb0004a40 comp rx=0 tx=0).stop 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 shutdown_connections 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddc4e540 0x564eddc50960 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc3e780 0x564eddc3c110 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5b96ffd640 0 client.16161 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3576061023 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc3b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 >> 172.21.15.33:0/3576061023 conn(0x564eddc2f9b0 msgr2=0x564eddc31700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 shutdown_connections 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3576061023 wait complete. 2024-01-25T16:17:13.983 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.974+0000 7f5bc03f0740 20 client.16161 trim_cache size 0 max 16384 2024-01-25T16:17:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 Processor -- start 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 -- start start 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 0x564eddc3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddc309b0 con 0x564eddc35090 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddc4cc40 con 0x564eddc359b0 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddc30be0 con 0x564eddc3e780 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bb6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bb6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58100/0 (socket says 172.21.15.33:58100) 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.987+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/3018827860 learned_addr learned my addr 172.21.15.33:0/3018827860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 0x564eddc3bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/3018827860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 msgr2=0x564eddc3bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 0x564eddc3bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/3018827860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 msgr2=0x564eddc3a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 -- 172.21.15.33:0/3018827860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eddb38060 con 0x564eddc35090 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 0x564eddc3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8095ef0 tx=0x7f5ba8069b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a16690fc23e4d82 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5b9d7fa640 1 -- 172.21.15.33:0/3018827860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80999f0 con 0x564eddc35090 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5b9d7fa640 1 -- 172.21.15.33:0/3018827860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ba809db40 con 0x564eddc35090 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.988+0000 7f5b9d7fa640 1 -- 172.21.15.33:0/3018827860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba80b9040 con 0x564eddc35090 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3018827860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 msgr2=0x564eddc3ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8095ef0 tx=0x7f5ba8069b20 comp rx=0 tx=0).stop 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3018827860 shutdown_connections 2024-01-25T16:17:13.995 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc3e780 0x564eddc3bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc359b0 0x564eddc3a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/3018827860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc35090 0x564eddc3ddc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3018827860 >> 172.21.15.33:0/3018827860 conn(0x564eddc2f9b0 msgr2=0x564eddc39f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3018827860 shutdown_connections 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- 172.21.15.33:0/3018827860 wait complete. 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 -- start start 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.989+0000 7f5bc03f0740 1 Processor -- start 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc456e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc3e780 0x564eddc44160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddc44830 con 0x564eddc359b0 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddc449a0 con 0x564eddc3e780 2024-01-25T16:17:13.996 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eddca4a90 con 0x564eddc35090 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 10 client.0 _ll_register_callbacks cb 0x7f5bac009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58114/0 (socket says 172.21.15.33:58114) 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 -- 172.21.15.33:0/2637471019 learned_addr learned my addr 172.21.15.33:0/2637471019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb77fe640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc3e780 0x564eddc44160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc456e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 -- 172.21.15.33:0/2637471019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 msgr2=0x564eddc456e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc456e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 -- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc3e780 msgr2=0x564eddc44160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc3e780 0x564eddc44160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 -- 172.21.15.33:0/2637471019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb81a63e0 con 0x564eddc359b0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bb67fc640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009930 tx=0x7f5bac02c1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fa3194f7b3e804 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5b96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac04e070 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5bac02ca50 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bac004040 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.990+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bac02cc10 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 10 client.16167 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x564eddb5da80 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bac05e080 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f5bac03c040 con 0x564eddc359b0 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 10 client.16167 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 20 client.16167 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddca5d20 0x564eddca8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 4 client.16167 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:17:13.998 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564edd89c600 con 0x564eddca5d20 2024-01-25T16:17:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.991+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddca5d20 0x564eddca8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.992+0000 7f5bb6ffd640 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddca5d20 0x564eddca8140 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a1a4a95a002083 server_cookie=7b4116946f604b19 in_seq=0 out_seq=0 2024-01-25T16:17:13.999 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:13.992+0000 7f5b96ffd640 10 client.16167 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5b96ffd640 1 -- 172.21.15.33:0/2637471019 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564edd89c600 con 0x564eddca5d20 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5b96ffd640 10 client.16167 handle_command_reply: tid=0 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 client.16167 shutdown 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 msgr2=0x564eddc43c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009930 tx=0x7f5bac02c1f0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 shutdown_connections 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x564eddca5d20 0x564eddca8140 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564eddc3e780 0x564eddc44160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564eddc359b0 0x564eddc43c20 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5b96ffd640 0 client.16167 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 --2- 172.21.15.33:0/2637471019 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x564eddc35090 0x564eddc456e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 >> 172.21.15.33:0/2637471019 conn(0x564eddc2f9b0 msgr2=0x564eddc31690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 shutdown_connections 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 1 -- 172.21.15.33:0/2637471019 wait complete. 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.086+0000 7f5bc03f0740 20 client.16167 trim_cache size 0 max 16384 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:17:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 msgr2=0x7f5ba00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 0x7f5ba00a39e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0005c40 tx=0x7f5bb0009200 comp rx=0 tx=0).stop 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 msgr2=0x7f5bb8072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d950 tx=0x7f5ba800de10 comp rx=0 tx=0).stop 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 shutdown_connections 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5ba00a1520 0x7f5ba00a39e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bb8073a70 0x7f5bb81a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bb8106000 0x7f5bb8073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 --2- 172.21.15.33:0/768841532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5bb8100f30 0x7f5bb8072ff0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 >> 172.21.15.33:0/768841532 conn(0x7f5bb80fc7d0 msgr2=0x7f5bb80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 shutdown_connections 2024-01-25T16:17:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:14.087+0000 7f5bbd832640 1 -- 172.21.15.33:0/768841532 wait complete. 2024-01-25T16:17:14.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 590599 7.77 MB/sec execute 371 sec latency 8681.281 ms 2024-01-25T16:17:15.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:15 smithi033 ceph-mon[31417]: pgmap v296: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:15.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:15 smithi152 ceph-mon[36015]: pgmap v296: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:15.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:15 smithi076 ceph-mon[35642]: pgmap v296: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 139 op/s 2024-01-25T16:17:15.782 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593128 7.80 MB/sec execute 372 sec latency 18.970 ms 2024-01-25T16:17:16.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593831 7.78 MB/sec execute 373 sec latency 75.668 ms 2024-01-25T16:17:17.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:17 smithi033 ceph-mon[31417]: pgmap v297: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:17.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:17 smithi152 ceph-mon[36015]: pgmap v297: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:17.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:17 smithi076 ceph-mon[35642]: pgmap v297: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:17.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596099 7.79 MB/sec execute 374 sec latency 18.397 ms 2024-01-25T16:17:18.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 597247 7.79 MB/sec execute 375 sec latency 39.662 ms 2024-01-25T16:17:19.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:19 smithi033 ceph-mon[31417]: pgmap v298: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:19.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:19 smithi152 ceph-mon[36015]: pgmap v298: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:19.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:19 smithi076 ceph-mon[35642]: pgmap v298: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:19.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 599209 7.79 MB/sec execute 376 sec latency 34.596 ms 2024-01-25T16:17:20.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600646 7.80 MB/sec execute 377 sec latency 88.201 ms 2024-01-25T16:17:21.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:21 smithi033 ceph-mon[31417]: pgmap v299: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:21.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:21 smithi152 ceph-mon[36015]: pgmap v299: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:21.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:21 smithi076 ceph-mon[35642]: pgmap v299: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.8 MiB/s wr, 129 op/s 2024-01-25T16:17:21.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 602551 7.80 MB/sec execute 378 sec latency 115.442 ms 2024-01-25T16:17:22.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 604022 7.81 MB/sec execute 379 sec latency 26.245 ms 2024-01-25T16:17:23.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:23 smithi033 ceph-mon[31417]: pgmap v300: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-01-25T16:17:23.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:23 smithi152 ceph-mon[36015]: pgmap v300: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-01-25T16:17:23.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:23 smithi076 ceph-mon[35642]: pgmap v300: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-01-25T16:17:23.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 605760 7.80 MB/sec execute 380 sec latency 16.198 ms 2024-01-25T16:17:24.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:24 smithi033 ceph-mon[31417]: pgmap v301: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-01-25T16:17:24.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:24 smithi152 ceph-mon[36015]: pgmap v301: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-01-25T16:17:24.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:24 smithi076 ceph-mon[35642]: pgmap v301: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-01-25T16:17:24.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 607549 7.82 MB/sec execute 381 sec latency 24.637 ms 2024-01-25T16:17:25.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 609300 7.81 MB/sec execute 382 sec latency 15.373 ms 2024-01-25T16:17:26.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611031 7.82 MB/sec execute 383 sec latency 92.687 ms 2024-01-25T16:17:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 -- 172.21.15.33:0/3037533595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c104bb0 msgr2=0x7f701c104f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3037533595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c104bb0 0x7f701c104f90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f700c009900 tx=0x7f700c02bc80 comp rx=0 tx=0).stop 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 -- 172.21.15.33:0/3037533595 shutdown_connections 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3037533595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c1007e0 0x7f701c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3037533595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c0ffe40 0x7f701c1002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3037533595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c104bb0 0x7f701c104f90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 -- 172.21.15.33:0/3037533595 >> 172.21.15.33:0/3037533595 conn(0x7f701c076a50 msgr2=0x7f701c076e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 -- 172.21.15.33:0/3037533595 shutdown_connections 2024-01-25T16:17:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.176+0000 7f7020e6b640 1 -- 172.21.15.33:0/3037533595 wait complete. 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 Processor -- start 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 -- start start 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 0x7f701c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c104bb0 0x7f701c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19c0c0 con 0x7f701c1007e0 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19c230 con 0x7f701c0ffe40 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7020e6b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701c19c3f0 con 0x7f701c104bb0 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57964/0 (socket says 172.21.15.33:57964) 2024-01-25T16:17:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 -- 172.21.15.33:0/3518710844 learned_addr learned my addr 172.21.15.33:0/3518710844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f701a59c640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 0x7f701c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f701ad9d640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c104bb0 0x7f701c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 -- 172.21.15.33:0/3518710844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c104bb0 msgr2=0x7f701c19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c104bb0 0x7f701c19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 -- 172.21.15.33:0/3518710844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 msgr2=0x7f701c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 0x7f701c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 -- 172.21.15.33:0/3518710844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c009560 con 0x7f701c1007e0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f701a59c640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 0x7f701c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.177+0000 7f7019d9b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f701000d770 tx=0x7f701000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.178+0000 7f700b7fe640 1 -- 172.21.15.33:0/3518710844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010009c30 con 0x7f701c1007e0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.178+0000 7f700b7fe640 1 -- 172.21.15.33:0/3518710844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f701000de80 con 0x7f701c1007e0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.178+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701c19c6d0 con 0x7f701c1007e0 2024-01-25T16:17:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.178+0000 7f700b7fe640 1 -- 172.21.15.33:0/3518710844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010004de0 con 0x7f701c1007e0 2024-01-25T16:17:27.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.178+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a3f50 con 0x7f701c1007e0 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f700b7fe640 1 -- 172.21.15.33:0/3518710844 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7010004f40 con 0x7f701c1007e0 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 Processor -- start 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 -- start start 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937b6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55909371c070 con 0x5590937ad090 2024-01-25T16:17:27.191 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559093719760 con 0x5590937b6780 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7023a31740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559093715050 con 0x5590937ad9b0 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701a59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57966/0 (socket says 172.21.15.33:57966) 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 -- 172.21.15.33:0/566076670 learned_addr learned my addr 172.21.15.33:0/566076670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f7019d9b640 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f700b7fe640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 0x7f70040a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 -- 172.21.15.33:0/566076670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 msgr2=0x5590937b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 -- 172.21.15.33:0/566076670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 msgr2=0x5590937b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701ad9d640 1 -- 172.21.15.33:0/566076670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55909370ce70 con 0x5590937ad090 2024-01-25T16:17:27.192 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.181+0000 7f701a59c640 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f701ad9d640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 0x7f70040a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f700b7fe640 1 -- 172.21.15.33:0/3518710844 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70100cc980 con 0x7f701c1007e0 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f701ad9d640 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7014011560 tx=0x7f7014011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba42a749f2f99d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f7008ff9640 1 -- 172.21.15.33:0/566076670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400f840 con 0x5590937ad090 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f7008ff9640 1 -- 172.21.15.33:0/566076670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f701400fe60 con 0x5590937ad090 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.182+0000 7f7008ff9640 1 -- 172.21.15.33:0/566076670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701400eae0 con 0x5590937ad090 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- 172.21.15.33:0/566076670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 msgr2=0x5590937ad470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7014011560 tx=0x7f7014011a20 comp rx=0 tx=0).stop 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f701ad9d640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 0x7f70040a39b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7014022000 tx=0x7f7014023000 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- 172.21.15.33:0/566076670 shutdown_connections 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.193 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- 172.21.15.33:0/566076670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937ad470 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- 172.21.15.33:0/566076670 >> 172.21.15.33:0/566076670 conn(0x5590937a79b0 msgr2=0x5590937a7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- 172.21.15.33:0/566076670 shutdown_connections 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- 172.21.15.33:0/566076670 wait complete. 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- start start 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 Processor -- start 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 0x5590937b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937b6780 0x5590937b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590936ce1e0 con 0x5590937ad090 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590936cde80 con 0x5590937ad9b0 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590936930c0 con 0x5590937b6780 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f7023a31740 10 client.0 _ll_register_callbacks cb 0x5590936b8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f701a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.194 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f701a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57968/0 (socket says 172.21.15.33:57968) 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.183+0000 7f701a59c640 1 -- 172.21.15.33:0/2290051390 learned_addr learned my addr 172.21.15.33:0/2290051390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7019d9b640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 0x5590937b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701ad9d640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937b6780 0x5590937b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 -- 172.21.15.33:0/2290051390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937b6780 msgr2=0x5590937b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937b6780 0x5590937b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 -- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 msgr2=0x5590937b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 0x5590937b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 -- 172.21.15.33:0/2290051390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7014009cc0 con 0x5590937ad090 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7019d9b640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 0x5590937b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f701a59c640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f700c035000 tx=0x7f700c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84c4c15d1000a98 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c004e40 con 0x5590937ad090 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f700c046070 con 0x5590937ad090 2024-01-25T16:17:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55909370e040 con 0x5590937ad090 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f700c004a70 con 0x5590937ad090 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7023a31740 10 client.16179 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5590937136d0 con 0x5590937ad090 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.184+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f700c057080 con 0x5590937ad090 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f700c03c040 con 0x5590937ad090 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 10 client.16179 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 20 client.16179 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5590937c6610 0x5590937c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 4 client.16179 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55909370dd60 con 0x5590937c6610 2024-01-25T16:17:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7019d9b640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5590937c6610 0x5590937c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7001ffb640 10 client.16179 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.185+0000 7f7019d9b640 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5590937c6610 0x5590937c8a50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30310f3388f6741e server_cookie=d86c4868f85e17d6 in_seq=0 out_seq=0 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7001ffb640 1 -- 172.21.15.33:0/2290051390 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55909370dd60 con 0x5590937c6610 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7001ffb640 10 client.16179 handle_command_reply: tid=0 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 client.16179 shutdown 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 msgr2=0x5590937b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f700c035000 tx=0x7f700c004270 comp rx=0 tx=0).stop 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 shutdown_connections 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5590937c6610 0x5590937c8a50 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937b6780 0x5590937b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7001ffb640 0 client.16179 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937ad9b0 0x5590937b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 --2- 172.21.15.33:0/2290051390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937b36e0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 >> 172.21.15.33:0/2290051390 conn(0x5590937a79b0 msgr2=0x5590937a9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 shutdown_connections 2024-01-25T16:17:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 1 -- 172.21.15.33:0/2290051390 wait complete. 2024-01-25T16:17:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.186+0000 7f7023a31740 20 client.16179 trim_cache size 0 max 16384 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 Processor -- start 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 -- start start 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad090 0x5590937b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590937ae5e0 con 0x5590937ad9b0 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590937b8b70 con 0x5590937b6780 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7023a31740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590937a3270 con 0x5590937ad090 2024-01-25T16:17:27.207 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57972/0 (socket says 172.21.15.33:57972) 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 -- 172.21.15.33:0/4022071627 learned_addr learned my addr 172.21.15.33:0/4022071627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f701ad9d640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f701a59c640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad090 0x5590937b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 -- 172.21.15.33:0/4022071627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad090 msgr2=0x5590937b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad090 0x5590937b5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 -- 172.21.15.33:0/4022071627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 msgr2=0x5590937b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.208 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f7019d9b640 1 -- 172.21.15.33:0/4022071627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590936b0060 con 0x5590937ad9b0 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.201+0000 7f701ad9d640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7019d9b640 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f701000d740 tx=0x7f70100cee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e3a04f82d8c57a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7003fff640 1 -- 172.21.15.33:0/4022071627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7010004780 con 0x5590937ad9b0 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7003fff640 1 -- 172.21.15.33:0/4022071627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7010009d90 con 0x5590937ad9b0 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7003fff640 1 -- 172.21.15.33:0/4022071627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70100960a0 con 0x5590937ad9b0 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 -- 172.21.15.33:0/4022071627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 msgr2=0x5590937b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f701000d740 tx=0x7f70100cee50 comp rx=0 tx=0).stop 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 -- 172.21.15.33:0/4022071627 shutdown_connections 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x5590937b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad9b0 0x5590937b3f80 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 --2- 172.21.15.33:0/4022071627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad090 0x5590937b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 -- 172.21.15.33:0/4022071627 >> 172.21.15.33:0/4022071627 conn(0x5590937a79b0 msgr2=0x5590937b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 -- 172.21.15.33:0/4022071627 shutdown_connections 2024-01-25T16:17:27.209 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.202+0000 7f7023a31740 1 -- 172.21.15.33:0/4022071627 wait complete. 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 -- start start 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 Processor -- start 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937bcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x559093816bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559093817100 con 0x5590937ad090 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559093817240 con 0x5590937b6780 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559093817400 con 0x5590937ad9b0 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7023a31740 10 client.0 _ll_register_callbacks cb 0x7f7014009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57988/0 (socket says 172.21.15.33:57988) 2024-01-25T16:17:27.210 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 -- 172.21.15.33:0/1215462710 learned_addr learned my addr 172.21.15.33:0/1215462710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7019d9b640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937bcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701ad9d640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x559093816bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 -- 172.21.15.33:0/1215462710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 msgr2=0x5590937bcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937bcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 -- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 msgr2=0x559093816bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x559093816bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 -- 172.21.15.33:0/1215462710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701c1a3f50 con 0x5590937ad090 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701ad9d640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x559093816bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f701a59c640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f700c000c00 tx=0x7f700c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d94a7c55ef426 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.203+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700c004dd0 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f700c046040 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f701400fe60 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f700c004a70 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 10 client.16185 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5590936d5a80 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f701400fe60 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f700c03c040 con 0x5590937ad090 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 10 client.16185 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 20 client.16185 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55909381eaf0 0x559093820f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 4 client.16185 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559093414600 con 0x55909381eaf0 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.204+0000 7f7019d9b640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55909381eaf0 0x559093820f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.205+0000 7f7019d9b640 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55909381eaf0 0x559093820f10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bf0f063feaa5fdf server_cookie=5e9d129eb9f0269c in_seq=0 out_seq=0 2024-01-25T16:17:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.205+0000 7f7001ffb640 10 client.16185 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7001ffb640 1 -- 172.21.15.33:0/1215462710 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559093414600 con 0x55909381eaf0 2024-01-25T16:17:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7001ffb640 10 client.16185 handle_command_reply: tid=0 2024-01-25T16:17:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 client.16185 shutdown 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 msgr2=0x5590937bc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f700c000c00 tx=0x7f700c0042e0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 shutdown_connections 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55909381eaf0 0x559093820f10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590937b6780 0x559093816bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5590937ad9b0 0x5590937bcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7001ffb640 0 client.16185 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 --2- 172.21.15.33:0/1215462710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5590937ad090 0x5590937bc7a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.309+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 >> 172.21.15.33:0/1215462710 conn(0x5590937a79b0 msgr2=0x5590937a61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 shutdown_connections 2024-01-25T16:17:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7023a31740 1 -- 172.21.15.33:0/1215462710 wait complete. 2024-01-25T16:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7023a31740 20 client.16185 trim_cache size 0 max 16384 2024-01-25T16:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:17:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 msgr2=0x7f70040a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 0x7f70040a39b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7014022000 tx=0x7f7014023000 comp rx=0 tx=0).stop 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 msgr2=0x7f701c197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f701000d770 tx=0x7f701000dc30 comp rx=0 tx=0).stop 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 shutdown_connections 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f70040a14f0 0x7f70040a39b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f701c104bb0 0x7f701c19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f701c1007e0 0x7f701c197790 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 --2- 172.21.15.33:0/3518710844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f701c0ffe40 0x7f701c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 >> 172.21.15.33:0/3518710844 conn(0x7f701c076a50 msgr2=0x7f701c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 shutdown_connections 2024-01-25T16:17:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:27.310+0000 7f7020e6b640 1 -- 172.21.15.33:0/3518710844 wait complete. 2024-01-25T16:17:27.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:27 smithi033 ceph-mon[31417]: pgmap v302: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-01-25T16:17:27.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:27 smithi152 ceph-mon[36015]: pgmap v302: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-01-25T16:17:27.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:27 smithi076 ceph-mon[35642]: pgmap v302: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 258 op/s 2024-01-25T16:17:27.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 612347 7.82 MB/sec execute 384 sec latency 48.284 ms 2024-01-25T16:17:28.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 614099 7.82 MB/sec execute 385 sec latency 289.654 ms 2024-01-25T16:17:29.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:29 smithi033 ceph-mon[31417]: pgmap v303: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:29.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:29 smithi152 ceph-mon[36015]: pgmap v303: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:29.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:29 smithi076 ceph-mon[35642]: pgmap v303: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:29.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615188 7.82 MB/sec execute 386 sec latency 292.393 ms 2024-01-25T16:17:30.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 617434 7.83 MB/sec execute 387 sec latency 16.141 ms 2024-01-25T16:17:31.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:31 smithi033 ceph-mon[31417]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:31.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:31 smithi152 ceph-mon[36015]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:31.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:31 smithi076 ceph-mon[35642]: pgmap v304: 97 pgs: 97 active+clean; 114 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-01-25T16:17:31.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 618384 7.82 MB/sec execute 388 sec latency 113.810 ms 2024-01-25T16:17:32.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:32 smithi033 ceph-mon[31417]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-01-25T16:17:32.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:32 smithi152 ceph-mon[36015]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-01-25T16:17:32.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:32 smithi076 ceph-mon[35642]: pgmap v305: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 334 op/s 2024-01-25T16:17:32.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 620361 7.82 MB/sec execute 389 sec latency 59.455 ms 2024-01-25T16:17:33.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 621919 7.83 MB/sec execute 390 sec latency 17.267 ms 2024-01-25T16:17:34.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:34 smithi033 ceph-mon[31417]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-01-25T16:17:34.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:34 smithi152 ceph-mon[36015]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-01-25T16:17:34.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:34 smithi076 ceph-mon[35642]: pgmap v306: 97 pgs: 97 active+clean; 103 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 244 op/s 2024-01-25T16:17:34.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 623928 7.83 MB/sec execute 391 sec latency 17.539 ms 2024-01-25T16:17:35.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 625412 7.84 MB/sec execute 392 sec latency 14.052 ms 2024-01-25T16:17:36.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 626971 7.84 MB/sec execute 393 sec latency 133.054 ms 2024-01-25T16:17:37.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:37 smithi033 ceph-mon[31417]: pgmap v307: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:17:37.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:37 smithi152 ceph-mon[36015]: pgmap v307: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:17:37.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:37 smithi076 ceph-mon[35642]: pgmap v307: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:17:37.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 628777 7.85 MB/sec execute 394 sec latency 30.551 ms 2024-01-25T16:17:38.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 630443 7.84 MB/sec execute 395 sec latency 18.831 ms 2024-01-25T16:17:39.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:39 smithi033 ceph-mon[31417]: pgmap v308: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:39.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:39 smithi152 ceph-mon[36015]: pgmap v308: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:39.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:39 smithi076 ceph-mon[35642]: pgmap v308: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:39.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 632392 7.86 MB/sec execute 396 sec latency 19.717 ms 2024-01-25T16:17:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 -- 172.21.15.33:0/1373088443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8104f00 msgr2=0x7f46f810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 --2- 172.21.15.33:0/1373088443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8104f00 0x7f46f810cad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009cd0 tx=0x7f46f002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.403 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 -- 172.21.15.33:0/1373088443 shutdown_connections 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 --2- 172.21.15.33:0/1373088443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8104f00 0x7f46f810cad0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 --2- 172.21.15.33:0/1373088443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f81007e0 0x7f46f8104820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 --2- 172.21.15.33:0/1373088443 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f80ffe30 0x7f46f8100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 -- 172.21.15.33:0/1373088443 >> 172.21.15.33:0/1373088443 conn(0x7f46f8076d20 msgr2=0x7f46f8077130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 -- 172.21.15.33:0/1373088443 shutdown_connections 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.402+0000 7f46fe256640 1 -- 172.21.15.33:0/1373088443 wait complete. 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 Processor -- start 2024-01-25T16:17:40.404 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 -- start start 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f80ffe30 0x7f46f81018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f8104f00 0x7f46f8101e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81a5930 con 0x7f46f8102340 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81a5aa0 con 0x7f46f80ffe30 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46fe256640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81a5c60 con 0x7f46f8104f00 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49738/0 (socket says 172.21.15.33:49738) 2024-01-25T16:17:40.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f80ffe30 0x7f46f81018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46f7fff640 1 -- 172.21.15.33:0/184800131 learned_addr learned my addr 172.21.15.33:0/184800131 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.403+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f8104f00 0x7f46f8101e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 -- 172.21.15.33:0/184800131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f8104f00 msgr2=0x7f46f8101e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f8104f00 0x7f46f8101e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 -- 172.21.15.33:0/184800131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f80ffe30 msgr2=0x7f46f81018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f80ffe30 0x7f46f81018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 -- 172.21.15.33:0/184800131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f0009900 con 0x7f46f8102340 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f7fff640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ca0 tx=0x7f46f0032550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f4ff9640 1 -- 172.21.15.33:0/184800131 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0032690 con 0x7f46f8102340 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f4ff9640 1 -- 172.21.15.33:0/184800131 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f0003970 con 0x7f46f8102340 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f81a5ee0 con 0x7f46f8102340 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46f4ff9640 1 -- 172.21.15.33:0/184800131 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00045e0 con 0x7f46f8102340 2024-01-25T16:17:40.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.404+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f81a6360 con 0x7f46f8102340 2024-01-25T16:17:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.406+0000 7f46f4ff9640 1 -- 172.21.15.33:0/184800131 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f46f003b040 con 0x7f46f8102340 2024-01-25T16:17:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f46f4ff9640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 0x7f46e00a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 Processor -- start 2024-01-25T16:17:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f46f77fe640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 0x7f46e00a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f46f4ff9640 1 -- 172.21.15.33:0/184800131 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f46f00ef7b0 con 0x7f46f8102340 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 -- start start 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc1129b0 0x55a1fc11b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 0x55a1fc11db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc081070 con 0x55a1fc11b780 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc07e760 con 0x55a1fc112090 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f4700e1a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc07a050 con 0x55a1fc1129b0 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.407+0000 7f46f77fe640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 0x7f46e00a39e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009bd0 tx=0x7f46e8005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 0x55a1fc11db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:56800/0 (socket says 172.21.15.33:56800) 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f77fe640 1 -- 172.21.15.33:0/275966668 learned_addr learned my addr 172.21.15.33:0/275966668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc1129b0 0x55a1fc11b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 -- 172.21.15.33:0/275966668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc1129b0 msgr2=0x55a1fc11b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc1129b0 0x55a1fc11b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 -- 172.21.15.33:0/275966668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 msgr2=0x55a1fc112470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 -- 172.21.15.33:0/275966668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1fc071e70 con 0x55a1fc11b780 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f77fe640 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46f7fff640 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 0x55a1fc11db70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ecfd0 tx=0x7f46f0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a6db548277fedd server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46ddffb640 1 -- 172.21.15.33:0/275966668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00c07d0 con 0x55a1fc11b780 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46ddffb640 1 -- 172.21.15.33:0/275966668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f00c4a10 con 0x55a1fc11b780 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.408+0000 7f46ddffb640 1 -- 172.21.15.33:0/275966668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00bc8e0 con 0x55a1fc11b780 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- 172.21.15.33:0/275966668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 msgr2=0x55a1fc11db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 0x55a1fc11db70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f46f00ecfd0 tx=0x7f46f0004040 comp rx=0 tx=0).stop 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- 172.21.15.33:0/275966668 shutdown_connections 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc11b780 0x55a1fc11db70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc1129b0 0x55a1fc11b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 --2- 172.21.15.33:0/275966668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc112470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- 172.21.15.33:0/275966668 >> 172.21.15.33:0/275966668 conn(0x55a1fc10c9b0 msgr2=0x55a1fc10cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- 172.21.15.33:0/275966668 shutdown_connections 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- 172.21.15.33:0/275966668 wait complete. 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 -- start start 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.409+0000 7f4700e1a740 1 Processor -- start 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 0x55a1fc1186b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc11b780 0x55a1fc119130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc0331e0 con 0x55a1fc1129b0 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc032e80 con 0x55a1fc11b780 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fbff80c0 con 0x55a1fc112090 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f4700e1a740 10 client.0 _ll_register_callbacks cb 0x55a1fc01dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 0x55a1fc1186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.417 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 0x55a1fc1186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:52414/0 (socket says 172.21.15.33:52414) 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f77fe640 1 -- 172.21.15.33:0/716453043 learned_addr learned my addr 172.21.15.33:0/716453043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f7fff640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc11b780 0x55a1fc119130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 -- 172.21.15.33:0/716453043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 msgr2=0x55a1fc1186b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 0x55a1fc1186b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 -- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc11b780 msgr2=0x55a1fc119130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc11b780 0x55a1fc119130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 -- 172.21.15.33:0/716453043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f81a6360 con 0x55a1fc1129b0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118bf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f46f8103350 tx=0x7f46ec012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b551df2d4ee192d5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.410+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec017070 con 0x55a1fc1129b0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1fc073040 con 0x55a1fc1129b0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f46ec004030 con 0x55a1fc1129b0 2024-01-25T16:17:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46ec017070 con 0x55a1fc1129b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 10 client.16197 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1fc0786d0 con 0x55a1fc1129b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a1fc073040 con 0x55a1fc1129b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f46ec022070 con 0x55a1fc1129b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 10 client.16197 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 20 client.16197 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc12b3b0 0x55a1fc12d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 4 client.16197 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1fc072d60 con 0x55a1fc12b3b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.411+0000 7f46f77fe640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc12b3b0 0x55a1fc12d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.412+0000 7f46d6ffd640 10 client.16197 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.412+0000 7f46f77fe640 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc12b3b0 0x55a1fc12d7d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc23e614c1200f3f server_cookie=743d88c316c710d8 in_seq=0 out_seq=0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.412+0000 7f46d6ffd640 1 -- 172.21.15.33:0/716453043 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1fc072d60 con 0x55a1fc12b3b0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.412+0000 7f46d6ffd640 10 client.16197 handle_command_reply: tid=0 2024-01-25T16:17:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.412+0000 7f4700e1a740 1 client.16197 shutdown 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 msgr2=0x55a1fc118bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118bf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f46f8103350 tx=0x7f46ec012b60 comp rx=0 tx=0).stop 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 shutdown_connections 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc12b3b0 0x55a1fc12d7d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc11b780 0x55a1fc119130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118bf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 --2- 172.21.15.33:0/716453043 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc112090 0x55a1fc1186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 >> 172.21.15.33:0/716453043 conn(0x55a1fc10c9b0 msgr2=0x55a1fc10e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f46d6ffd640 0 client.16197 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 shutdown_connections 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 1 -- 172.21.15.33:0/716453043 wait complete. 2024-01-25T16:17:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.413+0000 7f4700e1a740 20 client.16197 trim_cache size 0 max 16384 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.428+0000 7f4700e1a740 1 Processor -- start 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.428+0000 7f4700e1a740 1 -- start start 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc11ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc115910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc072d60 con 0x55a1fc1129b0 2024-01-25T16:17:40.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc073040 con 0x55a1fc112090 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f4700e1a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc11db70 con 0x55a1fc11b780 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49780/0 (socket says 172.21.15.33:49780) 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 -- 172.21.15.33:0/3994308986 learned_addr learned my addr 172.21.15.33:0/3994308986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f77fe640 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc11ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f7fff640 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc115910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 -- 172.21.15.33:0/3994308986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 msgr2=0x55a1fc115910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc115910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 -- 172.21.15.33:0/3994308986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 msgr2=0x55a1fc11ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc11ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 -- 172.21.15.33:0/3994308986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1fc015060 con 0x55a1fc1129b0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009d20 tx=0x7f46ec0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce3243d35356891 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.429+0000 7f46dd7fa640 1 -- 172.21.15.33:0/3994308986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec017040 con 0x55a1fc1129b0 2024-01-25T16:17:40.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f46dd7fa640 1 -- 172.21.15.33:0/3994308986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ec009e40 con 0x55a1fc1129b0 2024-01-25T16:17:40.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f46dd7fa640 1 -- 172.21.15.33:0/3994308986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec003880 con 0x55a1fc1129b0 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 -- 172.21.15.33:0/3994308986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 msgr2=0x55a1fc118f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009d20 tx=0x7f46ec0096c0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 -- 172.21.15.33:0/3994308986 shutdown_connections 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc1129b0 0x55a1fc118f50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 --2- 172.21.15.33:0/3994308986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc112090 0x55a1fc11ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 -- 172.21.15.33:0/3994308986 >> 172.21.15.33:0/3994308986 conn(0x55a1fc10c9b0 msgr2=0x55a1fc1139c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 -- 172.21.15.33:0/3994308986 shutdown_connections 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.430+0000 7f4700e1a740 1 -- 172.21.15.33:0/3994308986 wait complete. 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 -- start start 2024-01-25T16:17:40.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 Processor -- start 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 0x55a1fc120630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc1808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc120dd0 con 0x55a1fc112090 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc120f40 con 0x55a1fc1129b0 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1fc180df0 con 0x55a1fc11b780 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f4700e1a740 10 client.0 _ll_register_callbacks cb 0x7f46f00b82a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49782/0 (socket says 172.21.15.33:49782) 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f46f77fe640 1 -- 172.21.15.33:0/51129887 learned_addr learned my addr 172.21.15.33:0/51129887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 0x55a1fc120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.431+0000 7f46f7fff640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc1808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 -- 172.21.15.33:0/51129887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 msgr2=0x55a1fc1808b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc1808b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 -- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 msgr2=0x55a1fc120630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 0x55a1fc120630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 -- 172.21.15.33:0/51129887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1fc0786d0 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 0x55a1fc120630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46f77fe640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55a1fc12e0b0 tx=0x7f46e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42c5fbc29d49c87 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8024040 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f46e8029870 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46f00c4a10 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e8005020 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f4700e1a740 10 client.16203 fetch_fsmap learned FSMap version 32 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55a1fc03aa80 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46f00c4a10 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.432+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f46e801e420 con 0x55a1fc112090 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 10 client.16203 fetch_fsmap finished waiting for FSMap version 32 2024-01-25T16:17:40.437 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 20 client.16203 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc182720 0x55a1fc184b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 4 client.16203 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a1fbd79600 con 0x55a1fc182720 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc182720 0x55a1fc184b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f46d6ffd640 10 client.16203 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:40.438 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.433+0000 7f46f6ffd640 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc182720 0x55a1fc184b40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a6e945a3a1b830 server_cookie=8d1114eb04a6a92c in_seq=0 out_seq=0 2024-01-25T16:17:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.543+0000 7f46d6ffd640 1 -- 172.21.15.33:0/51129887 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a1fbd79600 con 0x55a1fc182720 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.543+0000 7f46d6ffd640 10 client.16203 handle_command_reply: tid=0 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.543+0000 7f4700e1a740 1 client.16203 shutdown 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 msgr2=0x55a1fc1200f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55a1fc12e0b0 tx=0x7f46e8004730 comp rx=0 tx=0).stop 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 shutdown_connections 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a1fc182720 0x55a1fc184b40 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a1fc11b780 0x55a1fc1808b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a1fc1129b0 0x55a1fc120630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 --2- 172.21.15.33:0/51129887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a1fc112090 0x55a1fc1200f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 >> 172.21.15.33:0/51129887 conn(0x55a1fc10c9b0 msgr2=0x55a1fc10dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46d6ffd640 0 client.16203 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 shutdown_connections 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 1 -- 172.21.15.33:0/51129887 wait complete. 2024-01-25T16:17:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f4700e1a740 20 client.16203 trim_cache size 0 max 16384 2024-01-25T16:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:17:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 msgr2=0x7f46e00a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 0x7f46e00a39e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009bd0 tx=0x7f46e8005f90 comp rx=0 tx=0).stop 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 msgr2=0x7f46f81a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009ca0 tx=0x7f46f0032550 comp rx=0 tx=0).stop 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 shutdown_connections 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f46e00a1520 0x7f46e00a39e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f8102340 0x7f46f81a5360 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46f8104f00 0x7f46f8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 --2- 172.21.15.33:0/184800131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46f80ffe30 0x7f46f81018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.544+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 >> 172.21.15.33:0/184800131 conn(0x7f46f8076d20 msgr2=0x7f46f8103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.545+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 shutdown_connections 2024-01-25T16:17:40.548 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:40.545+0000 7f46fe256640 1 -- 172.21.15.33:0/184800131 wait complete. 2024-01-25T16:17:40.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 633825 7.85 MB/sec execute 397 sec latency 66.518 ms 2024-01-25T16:17:41.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:41 smithi033 ceph-mon[31417]: pgmap v309: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:41.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:41 smithi152 ceph-mon[36015]: pgmap v309: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:41.631 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:17:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:17:41.631 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:17:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:17:41.631 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:17:41 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/30360384 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558c3b975520 con 0x558c3b94b000 2024-01-25T16:17:41.631 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:17:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:17:41.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:41 smithi076 ceph-mon[35642]: pgmap v309: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 237 op/s 2024-01-25T16:17:41.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 635841 7.86 MB/sec execute 398 sec latency 28.175 ms 2024-01-25T16:17:42.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:42 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:17:42.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:42 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:17:42.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:42 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:17:42.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 637239 7.86 MB/sec execute 399 sec latency 45.658 ms 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: pgmap v310: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: osdmap e101: 12 total, 12 up, 12 in 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/3006917566,v1:172.21.15.152:6837/3006917566] up:boot 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:17:43.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:43 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: pgmap v310: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: osdmap e101: 12 total, 12 up, 12 in 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/3006917566,v1:172.21.15.152:6837/3006917566] up:boot 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:17:43.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:43 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:43.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: pgmap v310: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 3.7 MiB/s wr, 391 op/s 2024-01-25T16:17:43.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: osdmap e101: 12 total, 12 up, 12 in 2024-01-25T16:17:43.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/3006917566,v1:172.21.15.152:6837/3006917566] up:boot 2024-01-25T16:17:43.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:17:43.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:17:43.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:43 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:43.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 639362 7.87 MB/sec execute 400 sec latency 17.436 ms 2024-01-25T16:17:44.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 640835 7.87 MB/sec execute 401 sec latency 21.404 ms 2024-01-25T16:17:45.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:45 smithi033 ceph-mon[31417]: pgmap v312: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 284 op/s 2024-01-25T16:17:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:45 smithi152 ceph-mon[36015]: pgmap v312: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 284 op/s 2024-01-25T16:17:45.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:45 smithi076 ceph-mon[35642]: pgmap v312: 97 pgs: 97 active+clean; 99 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.8 MiB/s wr, 284 op/s 2024-01-25T16:17:45.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 642922 7.88 MB/sec execute 402 sec latency 20.735 ms 2024-01-25T16:17:46.783 INFO:tasks.workunit.client.0.smithi033.stdout: 1 644492 7.88 MB/sec execute 403 sec latency 17.894 ms 2024-01-25T16:17:47.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:47 smithi033 ceph-mon[31417]: pgmap v313: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:47.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:47 smithi152 ceph-mon[36015]: pgmap v313: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:47.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:47 smithi076 ceph-mon[35642]: pgmap v313: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:47.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 646478 7.89 MB/sec execute 404 sec latency 48.623 ms 2024-01-25T16:17:48.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 647990 7.89 MB/sec execute 405 sec latency 21.459 ms 2024-01-25T16:17:49.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:49 smithi033 ceph-mon[31417]: pgmap v314: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:49.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:49 smithi152 ceph-mon[36015]: pgmap v314: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:49.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:49 smithi076 ceph-mon[35642]: pgmap v314: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:49.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 650038 7.90 MB/sec execute 406 sec latency 15.384 ms 2024-01-25T16:17:50.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 651667 7.89 MB/sec execute 407 sec latency 14.793 ms 2024-01-25T16:17:51.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:51 smithi033 ceph-mon[31417]: pgmap v315: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:51.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:51 smithi152 ceph-mon[36015]: pgmap v315: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:51.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:51 smithi076 ceph-mon[35642]: pgmap v315: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.2 MiB/s wr, 270 op/s 2024-01-25T16:17:51.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 653597 7.91 MB/sec execute 408 sec latency 42.244 ms 2024-01-25T16:17:52.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 654998 7.90 MB/sec execute 409 sec latency 69.255 ms 2024-01-25T16:17:53.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:53 smithi033 ceph-mon[31417]: pgmap v316: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-01-25T16:17:53.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:53 smithi152 ceph-mon[36015]: pgmap v316: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 -- 172.21.15.33:0/303608001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c100800 msgr2=0x7ff94c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 --2- 172.21.15.33:0/303608001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c100800 0x7ff94c10ca20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff944009ca0 tx=0x7ff94402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 -- 172.21.15.33:0/303608001 shutdown_connections 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 --2- 172.21.15.33:0/303608001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c100800 0x7ff94c10ca20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 --2- 172.21.15.33:0/303608001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 --2- 172.21.15.33:0/303608001 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 0x7ff94c104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 -- 172.21.15.33:0/303608001 >> 172.21.15.33:0/303608001 conn(0x7ff94c076a40 msgr2=0x7ff94c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 -- 172.21.15.33:0/303608001 shutdown_connections 2024-01-25T16:17:53.633 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.632+0000 7ff953eee640 1 -- 172.21.15.33:0/303608001 wait complete. 2024-01-25T16:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 Processor -- start 2024-01-25T16:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 -- start start 2024-01-25T16:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c100800 0x7ff94c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 0x7ff94c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c19c070 con 0x7ff94c0ffe40 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c19c1e0 con 0x7ff94c100800 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff953eee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c19c3a0 con 0x7ff94c104b90 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48960/0 (socket says 172.21.15.33:48960) 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff951c8a640 1 -- 172.21.15.33:0/3995482986 learned_addr learned my addr 172.21.15.33:0/3995482986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff95248b640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 0x7ff94c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.633+0000 7ff951489640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c100800 0x7ff94c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 -- 172.21.15.33:0/3995482986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 msgr2=0x7ff94c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 0x7ff94c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 -- 172.21.15.33:0/3995482986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c100800 msgr2=0x7ff94c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c100800 0x7ff94c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 -- 172.21.15.33:0/3995482986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff944009900 con 0x7ff94c0ffe40 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff94800e9f0 tx=0x7ff94800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff942ffd640 1 -- 172.21.15.33:0/3995482986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948009720 con 0x7ff94c0ffe40 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff942ffd640 1 -- 172.21.15.33:0/3995482986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94800cd80 con 0x7ff94c0ffe40 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c19c680 con 0x7ff94c0ffe40 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff942ffd640 1 -- 172.21.15.33:0/3995482986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff948010430 con 0x7ff94c0ffe40 2024-01-25T16:17:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.634+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c1a3f00 con 0x7ff94c0ffe40 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.636+0000 7ff942ffd640 1 -- 172.21.15.33:0/3995482986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ff948005020 con 0x7ff94c0ffe40 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.636+0000 7ff956b8f740 1 Processor -- start 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 -- start start 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8763470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b87639b0 0x55d6b876c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86d2070 con 0x55d6b8763090 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86cf760 con 0x55d6b87639b0 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff956b8f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86cb050 con 0x55d6b876c780 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff942ffd640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 0x7ff9380a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b87639b0 0x55d6b876c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff95248b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8763470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:45394/0 (socket says 172.21.15.33:45394) 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 -- 172.21.15.33:0/2886526949 learned_addr learned my addr 172.21.15.33:0/2886526949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff95248b640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 0x7ff9380a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff942ffd640 1 -- 172.21.15.33:0/3995482986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff948014070 con 0x7ff94c0ffe40 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 -- 172.21.15.33:0/2886526949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b87639b0 msgr2=0x55d6b876c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b87639b0 0x55d6b876c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 -- 172.21.15.33:0/2886526949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 msgr2=0x55d6b8763470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8763470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.637+0000 7ff951489640 1 -- 172.21.15.33:0/2886526949 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b86c2e70 con 0x55d6b876c780 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff951489640 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009c90 tx=0x7ff93c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3ef109298bf616 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff95248b640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 0x7ff9380a3a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff944004af0 tx=0x7ff944005c50 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff937fff640 1 -- 172.21.15.33:0/2886526949 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c019630 con 0x55d6b876c780 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff937fff640 1 -- 172.21.15.33:0/2886526949 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff93c019c50 con 0x55d6b876c780 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff937fff640 1 -- 172.21.15.33:0/2886526949 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00e9c0 con 0x55d6b876c780 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 -- 172.21.15.33:0/2886526949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 msgr2=0x55d6b876eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009c90 tx=0x7ff93c011810 comp rx=0 tx=0).stop 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 -- 172.21.15.33:0/2886526949 shutdown_connections 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b876c780 0x55d6b876eb70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b87639b0 0x55d6b876c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 --2- 172.21.15.33:0/2886526949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8763470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.638+0000 7ff956b8f740 1 -- 172.21.15.33:0/2886526949 >> 172.21.15.33:0/2886526949 conn(0x55d6b875d9b0 msgr2=0x55d6b875dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- 172.21.15.33:0/2886526949 shutdown_connections 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- 172.21.15.33:0/2886526949 wait complete. 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- start start 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 Processor -- start 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b87696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b876a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86841e0 con 0x55d6b87639b0 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8683e80 con 0x55d6b876c780 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86490c0 con 0x55d6b8763090 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff956b8f740 10 client.0 _ll_register_callbacks cb 0x55d6b866eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff951489640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b87696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.645 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff95248b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b876a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff951489640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48982/0 (socket says 172.21.15.33:48982) 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.639+0000 7ff951489640 1 -- 172.21.15.33:0/3165536643 learned_addr learned my addr 172.21.15.33:0/3165536643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 -- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 msgr2=0x55d6b876a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b876a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 -- 172.21.15.33:0/3165536643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 msgr2=0x55d6b8769c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 -- 172.21.15.33:0/3165536643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93c0098f0 con 0x55d6b8763090 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff95248b640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b876a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951489640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b87696e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ca990 tx=0x7ff9480999a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67027beb8c23067b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480054d0 con 0x55d6b8763090 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6b86c4040 con 0x55d6b8763090 2024-01-25T16:17:53.646 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff94809a040 con 0x55d6b8763090 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9480b8170 con 0x55d6b8763090 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff956b8f740 10 client.25753 fetch_fsmap learned FSMap version 34 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d6b86c96d0 con 0x55d6b8763090 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.640+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9480a6080 con 0x55d6b8763090 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff94801f060 con 0x55d6b8763090 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 10 client.25753 fetch_fsmap finished waiting for FSMap version 34 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 20 client.25753 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b877c5d0 0x55d6b877ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 4 client.25753 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d6b86c3d60 con 0x55d6b877c5d0 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff951489640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b877c5d0 0x55d6b877ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff9357fa640 10 client.25753 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:53.647 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.641+0000 7ff951489640 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b877c5d0 0x55d6b877ea10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272d949a26386e00 server_cookie=9e9fb8c315b9e144 in_seq=0 out_seq=0 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff9357fa640 1 -- 172.21.15.33:0/3165536643 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d6b86c3d60 con 0x55d6b877c5d0 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff9357fa640 10 client.25753 handle_command_reply: tid=0 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 client.25753 shutdown 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 msgr2=0x55d6b87696e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b87696e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9480ca990 tx=0x7ff9480999a0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 shutdown_connections 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b877c5d0 0x55d6b877ea10 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b876a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff9357fa640 0 client.25753 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3165536643 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b87696e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 >> 172.21.15.33:0/3165536643 conn(0x55d6b875d9b0 msgr2=0x55d6b875f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 shutdown_connections 2024-01-25T16:17:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 1 -- 172.21.15.33:0/3165536643 wait complete. 2024-01-25T16:17:53.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.643+0000 7ff956b8f740 20 client.25753 trim_cache size 0 max 16384 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 Processor -- start 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 -- start start 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b876be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8766940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.658 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b86c3d90 con 0x55d6b87639b0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8757ed0 con 0x55d6b876c780 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff956b8f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b8779030 con 0x55d6b8763090 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48988/0 (socket says 172.21.15.33:48988) 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 -- 172.21.15.33:0/3217243302 learned_addr learned my addr 172.21.15.33:0/3217243302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b876be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff95248b640 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8766940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 -- 172.21.15.33:0/3217243302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 msgr2=0x55d6b876be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b876be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 -- 172.21.15.33:0/3217243302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 msgr2=0x55d6b8766940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8766940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 -- 172.21.15.33:0/3217243302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6b8666060 con 0x55d6b87639b0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.657+0000 7ff951489640 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098c0 tx=0x7ff93c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45e28acb20bc3bf server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.659 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff9377fe640 1 -- 172.21.15.33:0/3217243302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c005a90 con 0x55d6b87639b0 2024-01-25T16:17:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff9377fe640 1 -- 172.21.15.33:0/3217243302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff93c005650 con 0x55d6b87639b0 2024-01-25T16:17:53.660 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff9377fe640 1 -- 172.21.15.33:0/3217243302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c019690 con 0x55d6b87639b0 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 -- 172.21.15.33:0/3217243302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 msgr2=0x55d6b8769f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0098c0 tx=0x7ff93c009850 comp rx=0 tx=0).stop 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 -- 172.21.15.33:0/3217243302 shutdown_connections 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8766940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b87639b0 0x55d6b8769f80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.663 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3217243302 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b8763090 0x55d6b876be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 -- 172.21.15.33:0/3217243302 >> 172.21.15.33:0/3217243302 conn(0x55d6b875d9b0 msgr2=0x55d6b8767fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 -- 172.21.15.33:0/3217243302 shutdown_connections 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.658+0000 7ff956b8f740 1 -- 172.21.15.33:0/3217243302 wait complete. 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 -- start start 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 Processor -- start 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b87639b0 0x55d6b8772520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8772a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b87d2b30 con 0x55d6b8763090 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b87d2ca0 con 0x55d6b876c780 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6b87d2e60 con 0x55d6b87639b0 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff956b8f740 10 client.0 _ll_register_callbacks cb 0x7ff93c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:17:53.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff951c8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48998/0 (socket says 172.21.15.33:48998) 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff951c8a640 1 -- 172.21.15.33:0/3121244552 learned_addr learned my addr 172.21.15.33:0/3121244552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff951489640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b87639b0 0x55d6b8772520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.659+0000 7ff95248b640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8772a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 -- 172.21.15.33:0/3121244552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b87639b0 msgr2=0x55d6b8772520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b87639b0 0x55d6b8772520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 -- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 msgr2=0x55d6b8772a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8772a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 -- 172.21.15.33:0/3121244552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6b86c2e70 con 0x55d6b8763090 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff951c8a640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff948096540 tx=0x7ff948099ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0407f4474041e5 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9480b8170 con 0x55d6b8763090 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff94809a040 con 0x55d6b8763090 2024-01-25T16:17:53.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff93c019c50 con 0x55d6b8763090 2024-01-25T16:17:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94809a7c0 con 0x55d6b8763090 2024-01-25T16:17:53.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 10 client.16224 fetch_fsmap learned FSMap version 34 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55d6b868ba80 con 0x55d6b8763090 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff93c019c50 con 0x55d6b8763090 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) v1 ==== 7894+0+0 (secure 0 0 0) 0x7ff94801f040 con 0x55d6b8763090 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 10 client.16224 fetch_fsmap finished waiting for FSMap version 34 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 20 client.16224 populate_metadata read hostname 'smithi033' 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b87d42f0 0x55d6b87d6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:17:53.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 4 client.16224 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:17:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.660+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d6b83ca600 con 0x55d6b87d42f0 2024-01-25T16:17:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.661+0000 7ff951489640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b87d42f0 0x55d6b87d6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:17:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.661+0000 7ff951489640 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b87d42f0 0x55d6b87d6710 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=766432b2feea026d server_cookie=8cb8cc633e04de8 in_seq=0 out_seq=0 2024-01-25T16:17:53.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.661+0000 7ff9357fa640 10 client.16224 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:53.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:53 smithi076 ceph-mon[35642]: pgmap v316: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-01-25T16:17:53.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 656980 7.91 MB/sec execute 410 sec latency 125.683 ms 2024-01-25T16:17:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff9357fa640 1 -- 172.21.15.33:0/3121244552 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d6b83ca600 con 0x55d6b87d42f0 2024-01-25T16:17:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff9357fa640 10 client.16224 handle_command_reply: tid=0 2024-01-25T16:17:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 client.16224 shutdown 2024-01-25T16:17:53.825 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 msgr2=0x55d6b8771fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff948096540 tx=0x7ff948099ad0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 shutdown_connections 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55d6b87d42f0 0x55d6b87d6710 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d6b876c780 0x55d6b8772a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55d6b87639b0 0x55d6b8772520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff9357fa640 0 client.16224 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 --2- 172.21.15.33:0/3121244552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d6b8763090 0x55d6b8771fe0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 >> 172.21.15.33:0/3121244552 conn(0x55d6b875d9b0 msgr2=0x55d6b875ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 shutdown_connections 2024-01-25T16:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.824+0000 7ff956b8f740 1 -- 172.21.15.33:0/3121244552 wait complete. 2024-01-25T16:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff956b8f740 20 client.16224 trim_cache size 0 max 16384 2024-01-25T16:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 msgr2=0x7ff9380a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 0x7ff9380a3a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff944004af0 tx=0x7ff944005c50 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 msgr2=0x7ff94c1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff94800e9f0 tx=0x7ff94800edf0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 shutdown_connections 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ff9380a1570 0x7ff9380a3a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff94c104b90 0x7ff94c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff94c100800 0x7ff94c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 --2- 172.21.15.33:0/3995482986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff94c0ffe40 0x7ff94c1971b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.825+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 >> 172.21.15.33:0/3995482986 conn(0x7ff94c076a40 msgr2=0x7ff94c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.826+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 shutdown_connections 2024-01-25T16:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:17:53.826+0000 7ff953eee640 1 -- 172.21.15.33:0/3995482986 wait complete. 2024-01-25T16:17:54.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 658051 7.90 MB/sec execute 411 sec latency 146.236 ms 2024-01-25T16:17:55.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:55 smithi033 ceph-mon[31417]: pgmap v317: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-01-25T16:17:55.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:55 smithi152 ceph-mon[36015]: pgmap v317: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-01-25T16:17:55.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:55 smithi076 ceph-mon[35642]: pgmap v317: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.7 MiB/s wr, 153 op/s 2024-01-25T16:17:55.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 660472 7.92 MB/sec execute 412 sec latency 18.818 ms 2024-01-25T16:17:56.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 661456 7.91 MB/sec execute 413 sec latency 16.313 ms 2024-01-25T16:17:57.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:57 smithi033 ceph-mon[31417]: pgmap v318: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-01-25T16:17:57.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:57 smithi152 ceph-mon[36015]: pgmap v318: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-01-25T16:17:57.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:57 smithi076 ceph-mon[35642]: pgmap v318: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-01-25T16:17:57.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 663698 7.92 MB/sec execute 414 sec latency 97.537 ms 2024-01-25T16:17:58.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:58 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:58.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:17:58 smithi033 ceph-mon[31417]: pgmap v319: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:17:58.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:58 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:58.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:17:58 smithi152 ceph-mon[36015]: pgmap v319: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:17:58.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:58 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:17:58.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:17:58 smithi076 ceph-mon[35642]: pgmap v319: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:17:58.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664631 7.91 MB/sec execute 415 sec latency 45.026 ms 2024-01-25T16:17:59.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 666838 7.92 MB/sec execute 416 sec latency 19.079 ms 2024-01-25T16:18:00.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 667914 7.92 MB/sec execute 417 sec latency 23.787 ms 2024-01-25T16:18:01.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:01 smithi033 ceph-mon[31417]: pgmap v320: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:18:01.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:01 smithi152 ceph-mon[36015]: pgmap v320: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:18:01.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:01 smithi076 ceph-mon[35642]: pgmap v320: 97 pgs: 97 active+clean; 107 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-01-25T16:18:01.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 669326 7.92 MB/sec execute 418 sec latency 23.267 ms 2024-01-25T16:18:02.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 670915 7.92 MB/sec execute 419 sec latency 72.273 ms 2024-01-25T16:18:03.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:03 smithi033 ceph-mon[31417]: pgmap v321: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2024-01-25T16:18:03.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:03 smithi033 ceph-mon[31417]: osdmap e102: 12 total, 12 up, 12 in 2024-01-25T16:18:03.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:03 smithi152 ceph-mon[36015]: pgmap v321: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2024-01-25T16:18:03.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:03 smithi152 ceph-mon[36015]: osdmap e102: 12 total, 12 up, 12 in 2024-01-25T16:18:03.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:03 smithi076 ceph-mon[35642]: pgmap v321: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.7 MiB/s wr, 356 op/s 2024-01-25T16:18:03.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:03 smithi076 ceph-mon[35642]: osdmap e102: 12 total, 12 up, 12 in 2024-01-25T16:18:03.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 671419 7.91 MB/sec execute 420 sec latency 46.230 ms 2024-01-25T16:18:04.505 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:04 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:18:04.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:04 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:18:04.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:04 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:18:04.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 672813 7.90 MB/sec execute 421 sec latency 42.149 ms 2024-01-25T16:18:05.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:05 smithi033 ceph-mon[31417]: pgmap v323: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-01-25T16:18:05.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:05 smithi152 ceph-mon[36015]: pgmap v323: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-01-25T16:18:05.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:05 smithi076 ceph-mon[35642]: pgmap v323: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-01-25T16:18:05.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 674826 7.91 MB/sec execute 422 sec latency 65.016 ms 2024-01-25T16:18:06.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:06 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 675652 7.90 MB/sec execute 423 sec latency 93.134 ms 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:18:06.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:06.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:06 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.880 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.880 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.880 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.880 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:06.880 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi033 to 177.2M 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi033 to 185904230: error parsing value: Value '185904230' is below minimum 939524096 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: pgmap v324: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.881 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:06 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 -- 172.21.15.33:0/1639079341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9741018c0 msgr2=0x7fd974105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 --2- 172.21.15.33:0/1639079341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9741018c0 0x7fd974105900 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fd968009930 tx=0x7fd96802bc30 comp rx=0 tx=0).stop 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 -- 172.21.15.33:0/1639079341 shutdown_connections 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 --2- 172.21.15.33:0/1639079341 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974105fe0 0x7fd97410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 --2- 172.21.15.33:0/1639079341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9741018c0 0x7fd974105900 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 --2- 172.21.15.33:0/1639079341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd9741012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 -- 172.21.15.33:0/1639079341 >> 172.21.15.33:0/1639079341 conn(0x7fd9740fc7b0 msgr2=0x7fd9740febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 -- 172.21.15.33:0/1639079341 shutdown_connections 2024-01-25T16:18:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.922+0000 7fd978bce640 1 -- 172.21.15.33:0/1639079341 wait complete. 2024-01-25T16:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.923+0000 7fd978bce640 1 Processor -- start 2024-01-25T16:18:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.923+0000 7fd978bce640 1 -- start start 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd974072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974073a70 0x7fd9741a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741a59f0 con 0x7fd974105fe0 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741a5b60 con 0x7fd974100f10 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741a5d20 con 0x7fd974073a70 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd971d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd971d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39318/0 (socket says 172.21.15.33:39318) 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd974072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd971d9b640 1 -- 172.21.15.33:0/396111138 learned_addr learned my addr 172.21.15.33:0/396111138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd972d9d640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974073a70 0x7fd9741a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 -- 172.21.15.33:0/396111138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974073a70 msgr2=0x7fd9741a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974073a70 0x7fd9741a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 -- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 msgr2=0x7fd974073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 -- 172.21.15.33:0/396111138 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd968009560 con 0x7fd974100f10 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd971d9b640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd97259c640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd974072ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd9637fe640 1 -- 172.21.15.33:0/396111138 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96400ccc0 con 0x7fd974100f10 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9741a5fa0 con 0x7fd974100f10 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd9637fe640 1 -- 172.21.15.33:0/396111138 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96400ce20 con 0x7fd974100f10 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd9637fe640 1 -- 172.21.15.33:0/396111138 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640187c0 con 0x7fd974100f10 2024-01-25T16:18:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.924+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741a6430 con 0x7fd974100f10 2024-01-25T16:18:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.926+0000 7fd9637fe640 1 -- 172.21.15.33:0/396111138 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd964010430 con 0x7fd974100f10 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd9637fe640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 0x7fd95c0a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd97b78e740 1 Processor -- start 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd971d9b640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 0x7fd95c0a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd9637fe640 1 -- 172.21.15.33:0/396111138 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd964014070 con 0x7fd974100f10 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd97b78e740 1 -- start start 2024-01-25T16:18:06.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.927+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97b78e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d418070 con 0x56361d4a9090 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97b78e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d415760 con 0x56361d4b2780 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97b78e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d411050 con 0x56361d4a99b0 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39320/0 (socket says 172.21.15.33:39320) 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 -- 172.21.15.33:0/2461401082 learned_addr learned my addr 172.21.15.33:0/2461401082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd971d9b640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 0x7fd95c0a39e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd968006010 tx=0x7fd968005a20 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd972d9d640 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 -- 172.21.15.33:0/2461401082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 msgr2=0x56361d4b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 -- 172.21.15.33:0/2461401082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 msgr2=0x56361d4b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 -- 172.21.15.33:0/2461401082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361d408e70 con 0x56361d4a9090 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd97259c640 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a1ea0 tx=0x7fd964099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20c371977bd61d1 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd960ff9640 1 -- 172.21.15.33:0/2461401082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96409e190 con 0x56361d4a9090 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd960ff9640 1 -- 172.21.15.33:0/2461401082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9640a23d0 con 0x56361d4a9090 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.928+0000 7fd960ff9640 1 -- 172.21.15.33:0/2461401082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9640106e0 con 0x56361d4a9090 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- 172.21.15.33:0/2461401082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 msgr2=0x56361d4a9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd9640a1ea0 tx=0x7fd964099a60 comp rx=0 tx=0).stop 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- 172.21.15.33:0/2461401082 shutdown_connections 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2461401082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a9090 0x56361d4a9470 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- 172.21.15.33:0/2461401082 >> 172.21.15.33:0/2461401082 conn(0x56361d4a39b0 msgr2=0x56361d4a3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- 172.21.15.33:0/2461401082 shutdown_connections 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- 172.21.15.33:0/2461401082 wait complete. 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 -- start start 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.929+0000 7fd97b78e740 1 Processor -- start 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a9090 0x56361d4af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d3ca1e0 con 0x56361d4b2780 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d3c9e80 con 0x56361d4a9090 2024-01-25T16:18:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d38f0c0 con 0x56361d4a99b0 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 10 client.0 _ll_register_callbacks cb 0x56361d3b4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39334/0 (socket says 172.21.15.33:39334) 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 -- 172.21.15.33:0/2323967007 learned_addr learned my addr 172.21.15.33:0/2323967007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97259c640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a9090 0x56361d4af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd971d9b640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 -- 172.21.15.33:0/2323967007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 msgr2=0x56361d4afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 -- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a9090 msgr2=0x56361d4af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a9090 0x56361d4af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 -- 172.21.15.33:0/2323967007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741a6430 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd972d9d640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0117d0 tx=0x7fd96c011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4d818277aff527 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c011ee0 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361d40a040 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd96c00fb40 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.930+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd96c00fd00 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 10 client.16236 fetch_fsmap learned FSMap version 35 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56361d40f6d0 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56361d40a040 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd96c01b550 con 0x56361d4b2780 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 10 client.16236 fetch_fsmap finished waiting for FSMap version 35 2024-01-25T16:18:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 20 client.16236 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d4c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 4 client.16236 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361d409d60 con 0x56361d4c2670 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97259c640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d4c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd959ffb640 10 client.16236 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.931+0000 7fd97259c640 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d4c4a90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71ef5209662a505c server_cookie=29ad6f9d4c40486e in_seq=0 out_seq=0 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd959ffb640 1 -- 172.21.15.33:0/2323967007 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56361d409d60 con 0x56361d4c2670 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd959ffb640 10 client.16236 handle_command_reply: tid=0 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 client.16236 shutdown 2024-01-25T16:18:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 msgr2=0x56361d4b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0117d0 tx=0x7fd96c011c90 comp rx=0 tx=0).stop 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 shutdown_connections 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d4c4a90 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd959ffb640 0 client.16236 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4b01a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a99b0 0x56361d4afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2323967007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a9090 0x56361d4af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 >> 172.21.15.33:0/2323967007 conn(0x56361d4a39b0 msgr2=0x56361d4a5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 shutdown_connections 2024-01-25T16:18:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 1 -- 172.21.15.33:0/2323967007 wait complete. 2024-01-25T16:18:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.932+0000 7fd97b78e740 20 client.16236 trim_cache size 0 max 16384 2024-01-25T16:18:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.945+0000 7fd97b78e740 1 Processor -- start 2024-01-25T16:18:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 -- start start 2024-01-25T16:18:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a99b0 0x56361d4affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d409d60 con 0x56361d4b2780 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d40a040 con 0x56361d4a99b0 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97b78e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d4a8370 con 0x56361d4a9090 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39344/0 (socket says 172.21.15.33:39344) 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 -- 172.21.15.33:0/3925699050 learned_addr learned my addr 172.21.15.33:0/3925699050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97259c640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd971d9b640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a99b0 0x56361d4affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 -- 172.21.15.33:0/3925699050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 msgr2=0x56361d4b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 -- 172.21.15.33:0/3925699050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a99b0 msgr2=0x56361d4affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a99b0 0x56361d4affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 -- 172.21.15.33:0/3925699050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361d3ac060 con 0x56361d4b2780 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd97259c640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd972d9d640 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0114c0 tx=0x7fd96c0059f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bae88ca507618a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd95bfff640 1 -- 172.21.15.33:0/3925699050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c01b630 con 0x56361d4b2780 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd95bfff640 1 -- 172.21.15.33:0/3925699050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96c016070 con 0x56361d4b2780 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.946+0000 7fd95bfff640 1 -- 172.21.15.33:0/3925699050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c02b450 con 0x56361d4b2780 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- 172.21.15.33:0/3925699050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 msgr2=0x56361d4aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0114c0 tx=0x7fd96c0059f0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- 172.21.15.33:0/3925699050 shutdown_connections 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4b2780 0x56361d4aca00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4a99b0 0x56361d4affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 --2- 172.21.15.33:0/3925699050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- 172.21.15.33:0/3925699050 >> 172.21.15.33:0/3925699050 conn(0x56361d4a39b0 msgr2=0x56361d4ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- 172.21.15.33:0/3925699050 shutdown_connections 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- 172.21.15.33:0/3925699050 wait complete. 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 -- start start 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.947+0000 7fd97b78e740 1 Processor -- start 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d4b8bc0 con 0x56361d4a99b0 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d518b60 con 0x56361d4b2780 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361d518ca0 con 0x56361d4a9090 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 10 client.0 _ll_register_callbacks cb 0x7fd964095c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39354/0 (socket says 172.21.15.33:39354) 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 -- 172.21.15.33:0/2682571921 learned_addr learned my addr 172.21.15.33:0/2682571921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd972d9d640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97259c640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 -- 172.21.15.33:0/2682571921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 msgr2=0x56361d4b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.954 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 -- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 msgr2=0x56361d4b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 -- 172.21.15.33:0/2682571921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56361d40f6d0 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd971d9b640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd968036040 tx=0x7fd968004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9fd71cbf4821b88 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd959ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968003ac0 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd968047040 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9640a23d0 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd968003c20 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 10 client.16242 fetch_fsmap learned FSMap version 35 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.948+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x56361d3d1a80 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd9640a23d0 con 0x56361d4a99b0 2024-01-25T16:18:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fd96804e5a0 con 0x56361d4a99b0 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 10 client.16242 fetch_fsmap finished waiting for FSMap version 35 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 20 client.16242 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d51c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 4 client.16242 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56361d110600 con 0x56361d4c2670 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97259c640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d51c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd959ffb640 10 client.16242 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:06.949+0000 7fd97259c640 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d51c5e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6410c54501cd03d5 server_cookie=104ef3ac9d8407be in_seq=0 out_seq=0 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd959ffb640 1 -- 172.21.15.33:0/2682571921 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56361d110600 con 0x56361d4c2670 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd959ffb640 10 client.16242 handle_command_reply: tid=0 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 client.16242 shutdown 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 msgr2=0x56361d4b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd968036040 tx=0x7fd968004590 comp rx=0 tx=0).stop 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 shutdown_connections 2024-01-25T16:18:07.130 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56361d4c2670 0x56361d51c5e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd959ffb640 0 client.16242 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56361d4b2780 0x56361d4b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56361d4a99b0 0x56361d4b7da0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 --2- 172.21.15.33:0/2682571921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56361d4a9090 0x56361d4b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 >> 172.21.15.33:0/2682571921 conn(0x56361d4a39b0 msgr2=0x56361d4a21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 shutdown_connections 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 1 -- 172.21.15.33:0/2682571921 wait complete. 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.129+0000 7fd97b78e740 20 client.16242 trim_cache size 0 max 16384 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:18:07.131 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:18:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:18:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:18:07.132 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 msgr2=0x7fd95c0a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 0x7fd95c0a39e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd968006010 tx=0x7fd968005a20 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 msgr2=0x7fd974072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd974072ff0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd96400e9f0 tx=0x7fd96400eeb0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 shutdown_connections 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd95c0a1520 0x7fd95c0a39e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd974073a70 0x7fd9741a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd974105fe0 0x7fd974073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 --2- 172.21.15.33:0/396111138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd974100f10 0x7fd974072ff0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 >> 172.21.15.33:0/396111138 conn(0x7fd9740fc7b0 msgr2=0x7fd9740fea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 shutdown_connections 2024-01-25T16:18:07.133 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:07.130+0000 7fd978bce640 1 -- 172.21.15.33:0/396111138 wait complete. 2024-01-25T16:18:07.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 677168 7.90 MB/sec execute 424 sec latency 109.702 ms 2024-01-25T16:18:07.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:07 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:18:07.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:07 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:18:08.062 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:07 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:18:08.062 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:07 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:18:08.130 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:07 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi152 to 174.1M 2024-01-25T16:18:08.130 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:07 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi152 to 182637056: error parsing value: Value '182637056' is below minimum 939524096 2024-01-25T16:18:08.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 678427 7.90 MB/sec execute 425 sec latency 106.877 ms 2024-01-25T16:18:08.916 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:08 smithi076 ceph-mon[35642]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:09.062 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:08 smithi033 ceph-mon[31417]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:09.130 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:08 smithi152 ceph-mon[36015]: pgmap v325: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:09.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 679744 7.90 MB/sec execute 426 sec latency 57.263 ms 2024-01-25T16:18:10.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 680913 7.89 MB/sec execute 427 sec latency 85.253 ms 2024-01-25T16:18:11.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:11 smithi033 ceph-mon[31417]: pgmap v326: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:11.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:11 smithi033 ceph-mon[31417]: osdmap e103: 12 total, 12 up, 12 in 2024-01-25T16:18:11.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:11 smithi152 ceph-mon[36015]: pgmap v326: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:11.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:11 smithi152 ceph-mon[36015]: osdmap e103: 12 total, 12 up, 12 in 2024-01-25T16:18:11.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:11 smithi076 ceph-mon[35642]: pgmap v326: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 233 op/s 2024-01-25T16:18:11.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:11 smithi076 ceph-mon[35642]: osdmap e103: 12 total, 12 up, 12 in 2024-01-25T16:18:11.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 682213 7.90 MB/sec execute 428 sec latency 57.258 ms 2024-01-25T16:18:12.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:12 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:12.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:12 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:12 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:12 smithi033 ceph-mon[31417]: pgmap v328: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:12.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:12 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:12.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:12 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:12 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:12 smithi152 ceph-mon[36015]: pgmap v328: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:12.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:12 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:12.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:12 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:12 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:12.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:12 smithi076 ceph-mon[35642]: pgmap v328: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:12.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 683552 7.89 MB/sec execute 429 sec latency 86.309 ms 2024-01-25T16:18:13.416 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:13.416 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.416 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.416 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:18:13.417 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:18:13.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-01-25T16:18:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: Adjusting osd_memory_target on smithi076 to 174.1M 2024-01-25T16:18:13.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: Unable to set osd_memory_target on smithi076 to 182634905: error parsing value: Value '182634905' is below minimum 939524096 2024-01-25T16:18:13.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:18:13.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:18:13.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:18:13.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 685533 7.90 MB/sec execute 430 sec latency 31.320 ms 2024-01-25T16:18:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:14 smithi033 ceph-mon[31417]: pgmap v329: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:14.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:14 smithi152 ceph-mon[36015]: pgmap v329: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:14.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:14 smithi076 ceph-mon[35642]: pgmap v329: 97 pgs: 97 active+clean; 103 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 257 op/s 2024-01-25T16:18:14.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 686919 7.90 MB/sec execute 431 sec latency 64.926 ms 2024-01-25T16:18:15.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 688955 7.91 MB/sec execute 432 sec latency 118.388 ms 2024-01-25T16:18:16.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 690161 7.90 MB/sec execute 433 sec latency 16.787 ms 2024-01-25T16:18:17.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:17 smithi033 ceph-mon[31417]: pgmap v330: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:17.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:17 smithi152 ceph-mon[36015]: pgmap v330: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:17.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:17 smithi076 ceph-mon[35642]: pgmap v330: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:17.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 692475 7.92 MB/sec execute 434 sec latency 51.458 ms 2024-01-25T16:18:18.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 693405 7.90 MB/sec execute 435 sec latency 18.222 ms 2024-01-25T16:18:19.562 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:19 smithi033 ceph-mon[31417]: pgmap v331: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:19.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:19 smithi152 ceph-mon[36015]: pgmap v331: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:19.668 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:19 smithi076 ceph-mon[35642]: pgmap v331: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:19.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 696027 7.92 MB/sec execute 436 sec latency 19.655 ms 2024-01-25T16:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- 172.21.15.33:0/500384772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 msgr2=0x7f5f84101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/500384772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84101310 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009930 tx=0x7f5f8002bc30 comp rx=0 tx=0).stop 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- 172.21.15.33:0/500384772 shutdown_connections 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/500384772 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 0x7f5f8410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/500384772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f841018e0 0x7f5f84105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/500384772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84101310 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- 172.21.15.33:0/500384772 >> 172.21.15.33:0/500384772 conn(0x7f5f840fc7d0 msgr2=0x7f5f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- 172.21.15.33:0/500384772 shutdown_connections 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- 172.21.15.33:0/500384772 wait complete. 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 Processor -- start 2024-01-25T16:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.225+0000 7f5f8c21c640 1 -- start start 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 0x7f5f84197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 0x7f5f8419bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419c480 con 0x7f5f84100f30 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419c5f0 con 0x7f5f8419ba90 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8c21c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8419c760 con 0x7f5f84106000 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60398/0 (socket says 172.21.15.33:60398) 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 -- 172.21.15.33:0/2245578668 learned_addr learned my addr 172.21.15.33:0/2245578668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 0x7f5f8419bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f897b7640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 0x7f5f84197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.230 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 -- 172.21.15.33:0/2245578668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 msgr2=0x7f5f84197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 0x7f5f84197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 -- 172.21.15.33:0/2245578668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 msgr2=0x7f5f8419bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 0x7f5f8419bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 -- 172.21.15.33:0/2245578668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009560 con 0x7f5f84100f30 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 0x7f5f8419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.226+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005940 tx=0x7f5f80004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.227+0000 7f5f7affd640 1 -- 172.21.15.33:0/2245578668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004250 con 0x7f5f84100f30 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.227+0000 7f5f7affd640 1 -- 172.21.15.33:0/2245578668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f800043b0 con 0x7f5f84100f30 2024-01-25T16:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.227+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f841a3d40 con 0x7f5f84100f30 2024-01-25T16:18:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.227+0000 7f5f7affd640 1 -- 172.21.15.33:0/2245578668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80003d00 con 0x7f5f84100f30 2024-01-25T16:18:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.227+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f841a41c0 con 0x7f5f84100f30 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.229+0000 7f5f7affd640 1 -- 172.21.15.33:0/2245578668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f5f80005000 con 0x7f5f84100f30 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.229+0000 7f5f8edde740 1 Processor -- start 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 -- start start 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f7affd640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 0x7f5f700a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e619070 con 0x557c8e6aa9b0 2024-01-25T16:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e616760 con 0x557c8e6b3780 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8edde740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e612050 con 0x557c8e6aa090 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 0x7f5f700a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f7affd640 1 -- 172.21.15.33:0/2245578668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5f800ef5e0 con 0x7f5f84100f30 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60402/0 (socket says 172.21.15.33:60402) 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 -- 172.21.15.33:0/1982516197 learned_addr learned my addr 172.21.15.33:0/1982516197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 -- 172.21.15.33:0/1982516197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 msgr2=0x557c8e6aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 -- 172.21.15.33:0/1982516197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 msgr2=0x557c8e6b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 -- 172.21.15.33:0/1982516197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8e609e70 con 0x557c8e6aa9b0 2024-01-25T16:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009c60 tx=0x7f5f74011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e9b82002ca009e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.230+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 0x7f5f700a39b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x557c8e6a5dd0 tx=0x7f5f7c005bd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f6ffff640 1 -- 172.21.15.33:0/1982516197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74004230 con 0x557c8e6aa9b0 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f6ffff640 1 -- 172.21.15.33:0/1982516197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f74004390 con 0x557c8e6aa9b0 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f6ffff640 1 -- 172.21.15.33:0/1982516197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f74004cd0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 -- 172.21.15.33:0/1982516197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 msgr2=0x557c8e6b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009c60 tx=0x7f5f74011810 comp rx=0 tx=0).stop 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 -- 172.21.15.33:0/1982516197 shutdown_connections 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b3240 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1982516197 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 -- 172.21.15.33:0/1982516197 >> 172.21.15.33:0/1982516197 conn(0x557c8e6a49b0 msgr2=0x557c8e6a4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 -- 172.21.15.33:0/1982516197 shutdown_connections 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.231+0000 7f5f8edde740 1 -- 172.21.15.33:0/1982516197 wait complete. 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 -- start start 2024-01-25T16:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 Processor -- start 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e5cb1e0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e5cae80 con 0x557c8e6b3780 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e5900c0 con 0x557c8e6aa090 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8edde740 10 client.0 _ll_register_callbacks cb 0x557c8e5b5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f897b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f897b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60412/0 (socket says 172.21.15.33:60412) 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f897b7640 1 -- 172.21.15.33:0/1125798980 learned_addr learned my addr 172.21.15.33:0/1125798980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.232+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 -- 172.21.15.33:0/1125798980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 msgr2=0x557c8e6b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6b06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 -- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 msgr2=0x557c8e6b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 -- 172.21.15.33:0/1125798980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f74009890 con 0x557c8e6aa9b0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f74011d10 tx=0x7f5f74019ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d404e6f1cba7cdc server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7400e7c0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5f740046c0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c8e60b040 con 0x557c8e6aa9b0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f74009ed0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f8edde740 10 client.16254 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:20.244 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.233+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557c8e6106d0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557c8e60b040 con 0x557c8e6aa9b0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f740249b0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 10 client.16254 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 20 client.16254 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e6c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 4 client.16254 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c8e60ad60 con 0x557c8e6c3580 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e6c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f6d7fa640 10 client.16254 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.235+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e6c59a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7526cfd8b9cf931 server_cookie=3ba2674b57ce3ecb in_seq=0 out_seq=0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.236+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/1125798980 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c8e60ad60 con 0x557c8e6c3580 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.236+0000 7f5f6d7fa640 10 client.16254 handle_command_reply: tid=0 2024-01-25T16:18:20.245 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 client.16254 shutdown 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 msgr2=0x557c8e6b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5f74011d10 tx=0x7f5f74019ba0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 shutdown_connections 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e6c59a0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6b3780 0x557c8e6b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b0c20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f6d7fa640 0 client.16254 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1125798980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa090 0x557c8e6b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 >> 172.21.15.33:0/1125798980 conn(0x557c8e6a49b0 msgr2=0x557c8e6a6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 shutdown_connections 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 1 -- 172.21.15.33:0/1125798980 wait complete. 2024-01-25T16:18:20.246 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.237+0000 7f5f8edde740 20 client.16254 trim_cache size 0 max 16384 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 Processor -- start 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 -- start start 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 0x557c8e6aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6b3780 0x557c8e6abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e60b040 con 0x557c8e6aa090 2024-01-25T16:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e6a57b0 con 0x557c8e6aa9b0 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8edde740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e6b16b0 con 0x557c8e6b3780 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f89fb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f89fb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60418/0 (socket says 172.21.15.33:60418) 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f89fb8640 1 -- 172.21.15.33:0/1560506117 learned_addr learned my addr 172.21.15.33:0/1560506117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 0x557c8e6aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.252+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6b3780 0x557c8e6abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 -- 172.21.15.33:0/1560506117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6b3780 msgr2=0x557c8e6abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6b3780 0x557c8e6abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 -- 172.21.15.33:0/1560506117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 msgr2=0x557c8e6aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 0x557c8e6aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 -- 172.21.15.33:0/1560506117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c8e5ad060 con 0x557c8e6aa090 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f897b7640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 0x557c8e6aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5f800b89b0 tx=0x7f5f800a7040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df2236e50208c48 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f6f7fe640 1 -- 172.21.15.33:0/1560506117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800035f0 con 0x557c8e6aa090 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f6f7fe640 1 -- 172.21.15.33:0/1560506117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f800bcbb0 con 0x557c8e6aa090 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.253+0000 7f5f6f7fe640 1 -- 172.21.15.33:0/1560506117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800ba850 con 0x557c8e6aa090 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- 172.21.15.33:0/1560506117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 msgr2=0x557c8e6b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5f800b89b0 tx=0x7f5f800a7040 comp rx=0 tx=0).stop 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- 172.21.15.33:0/1560506117 shutdown_connections 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6b3780 0x557c8e6abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa9b0 0x557c8e6aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- 172.21.15.33:0/1560506117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6aa090 0x557c8e6b2e10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- 172.21.15.33:0/1560506117 >> 172.21.15.33:0/1560506117 conn(0x557c8e6a49b0 msgr2=0x557c8e6aaf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- 172.21.15.33:0/1560506117 shutdown_connections 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- 172.21.15.33:0/1560506117 wait complete. 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- start start 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 Processor -- start 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 0x557c8e6b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e6badb0 con 0x557c8e6b3780 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e719aa0 con 0x557c8e6aa090 2024-01-25T16:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c8e719c60 con 0x557c8e6aa9b0 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.254+0000 7f5f8edde740 10 client.0 _ll_register_callbacks cb 0x7f5f740097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8a7b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 0x557c8e6b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8a7b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60420/0 (socket says 172.21.15.33:60420) 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8a7b9640 1 -- 172.21.15.33:0/120880249 learned_addr learned my addr 172.21.15.33:0/120880249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f897b7640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 -- 172.21.15.33:0/120880249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 msgr2=0x557c8e6b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 -- 172.21.15.33:0/120880249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 msgr2=0x557c8e6b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 -- 172.21.15.33:0/120880249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c8e6106d0 con 0x557c8e6aa090 2024-01-25T16:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f897b7640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8a7b9640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f89fb8640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 0x557c8e6b8e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009430 tx=0x7f5f800c4910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b85eb141c02e651 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800bc4b0 con 0x557c8e6aa090 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f74004390 con 0x557c8e6aa090 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5f800ba5f0 con 0x557c8e6aa090 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f800ba850 con 0x557c8e6aa090 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8edde740 10 client.25784 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.255+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f5f841a41c0 con 0x557c8e6aa090 2024-01-25T16:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f800d8070 con 0x557c8e6aa090 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5f8003c070 con 0x557c8e6aa090 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 10 client.25784 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 20 client.25784 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e71d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 4 client.25784 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c8e311600 con 0x557c8e6c3580 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f897b7640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e71d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f897b7640 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e71d120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f156dbed35bcdee server_cookie=5ecfd606429a993b in_seq=0 out_seq=0 2024-01-25T16:18:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.256+0000 7f5f6d7fa640 10 client.25784 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.351+0000 7f5f6d7fa640 1 -- 172.21.15.33:0/120880249 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c8e311600 con 0x557c8e6c3580 2024-01-25T16:18:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.351+0000 7f5f6d7fa640 10 client.25784 handle_command_reply: tid=0 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.351+0000 7f5f8edde740 1 client.25784 shutdown 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 msgr2=0x557c8e6b8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 0x557c8e6b8e10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009430 tx=0x7f5f800c4910 comp rx=0 tx=0).stop 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 shutdown_connections 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x557c8e6c3580 0x557c8e71d120 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f6d7fa640 0 client.25784 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557c8e6b3780 0x557c8e6b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x557c8e6aa9b0 0x557c8e6b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 --2- 172.21.15.33:0/120880249 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557c8e6aa090 0x557c8e6b8e10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 >> 172.21.15.33:0/120880249 conn(0x557c8e6a49b0 msgr2=0x557c8e6aba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 shutdown_connections 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 1 -- 172.21.15.33:0/120880249 wait complete. 2024-01-25T16:18:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.352+0000 7f5f8edde740 20 client.25784 trim_cache size 0 max 16384 2024-01-25T16:18:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:18:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:18:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:18:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 msgr2=0x7f5f700a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 0x7f5f700a39b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x557c8e6a5dd0 tx=0x7f5f7c005bd0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 msgr2=0x7f5f84197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005940 tx=0x7f5f80004bd0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 shutdown_connections 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f5f700a14f0 0x7f5f700a39b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8419ba90 0x7f5f8419bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f84106000 0x7f5f84197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 --2- 172.21.15.33:0/2245578668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f84100f30 0x7f5f84197220 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.353+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 >> 172.21.15.33:0/2245578668 conn(0x7f5f840fc7d0 msgr2=0x7f5f840feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.354+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 shutdown_connections 2024-01-25T16:18:20.357 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:20.354+0000 7f5f8c21c640 1 -- 172.21.15.33:0/2245578668 wait complete. 2024-01-25T16:18:20.784 INFO:tasks.workunit.client.0.smithi033.stdout: 1 696762 7.91 MB/sec execute 437 sec latency 45.734 ms 2024-01-25T16:18:21.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:21 smithi033 ceph-mon[31417]: pgmap v332: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:21.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:21 smithi152 ceph-mon[36015]: pgmap v332: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:21.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:21 smithi076 ceph-mon[35642]: pgmap v332: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-01-25T16:18:21.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 698939 7.92 MB/sec execute 438 sec latency 116.095 ms 2024-01-25T16:18:22.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 700157 7.92 MB/sec execute 439 sec latency 41.676 ms 2024-01-25T16:18:23.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:23 smithi033 ceph-mon[31417]: pgmap v333: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-01-25T16:18:23.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:23 smithi152 ceph-mon[36015]: pgmap v333: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-01-25T16:18:23.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:23 smithi076 ceph-mon[35642]: pgmap v333: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.2 MiB/s wr, 368 op/s 2024-01-25T16:18:23.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 702025 7.91 MB/sec execute 440 sec latency 35.262 ms 2024-01-25T16:18:24.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:24 smithi033 ceph-mon[31417]: pgmap v334: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:18:24.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:24 smithi152 ceph-mon[36015]: pgmap v334: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:18:24.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:24 smithi076 ceph-mon[35642]: pgmap v334: 97 pgs: 97 active+clean; 106 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:18:24.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 703558 7.92 MB/sec execute 441 sec latency 15.047 ms 2024-01-25T16:18:25.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 705572 7.92 MB/sec execute 442 sec latency 18.382 ms 2024-01-25T16:18:26.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 707087 7.93 MB/sec execute 443 sec latency 17.436 ms 2024-01-25T16:18:27.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:27 smithi033 ceph-mon[31417]: pgmap v335: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-01-25T16:18:27.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:27 smithi152 ceph-mon[36015]: pgmap v335: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-01-25T16:18:27.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:27 smithi076 ceph-mon[35642]: pgmap v335: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 322 op/s 2024-01-25T16:18:27.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 709114 7.93 MB/sec execute 444 sec latency 18.089 ms 2024-01-25T16:18:28.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:28 smithi033 ceph-mon[31417]: pgmap v336: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:28.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:28 smithi152 ceph-mon[36015]: pgmap v336: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:28.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:28 smithi076 ceph-mon[35642]: pgmap v336: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:28.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 710541 7.94 MB/sec execute 445 sec latency 57.394 ms 2024-01-25T16:18:29.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 712593 7.94 MB/sec execute 446 sec latency 16.539 ms 2024-01-25T16:18:30.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 714109 7.95 MB/sec execute 447 sec latency 59.695 ms 2024-01-25T16:18:31.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:31 smithi033 ceph-mon[31417]: pgmap v337: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:31.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:31 smithi152 ceph-mon[36015]: pgmap v337: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:31.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:31 smithi076 ceph-mon[35642]: pgmap v337: 97 pgs: 97 active+clean; 112 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 242 op/s 2024-01-25T16:18:31.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 716062 7.94 MB/sec execute 448 sec latency 24.310 ms 2024-01-25T16:18:32.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 717618 7.95 MB/sec execute 449 sec latency 14.886 ms 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 -- 172.21.15.33:0/2193890582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 msgr2=0x7f7e9c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 --2- 172.21.15.33:0/2193890582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0687d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009860 tx=0x7f7e9002bc00 comp rx=0 tx=0).stop 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 -- 172.21.15.33:0/2193890582 shutdown_connections 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 --2- 172.21.15.33:0/2193890582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c068d10 0x7f7e9c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 --2- 172.21.15.33:0/2193890582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0687d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 --2- 172.21.15.33:0/2193890582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c106de0 0x7f7e9c1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 -- 172.21.15.33:0/2193890582 >> 172.21.15.33:0/2193890582 conn(0x7f7e9c076a40 msgr2=0x7f7e9c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 -- 172.21.15.33:0/2193890582 shutdown_connections 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.449+0000 7f7ea203d640 1 -- 172.21.15.33:0/2193890582 wait complete. 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 Processor -- start 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 -- start start 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c106de0 0x7f7e9c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a54d0 con 0x7f7e9c068d10 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a5640 con 0x7f7e9c068370 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7ea203d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c1a5800 con 0x7f7e9c106de0 2024-01-25T16:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57530/0 (socket says 172.21.15.33:57530) 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7e9affd640 1 -- 172.21.15.33:0/1157766486 learned_addr learned my addr 172.21.15.33:0/1157766486 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.450+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c106de0 0x7f7e9c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1157766486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c106de0 msgr2=0x7f7e9c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c106de0 0x7f7e9c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 msgr2=0x7f7e9c100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1157766486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009560 con 0x7f7e9c068370 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0fff10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c005fd0 tx=0x7f7e8c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e98ff9640 1 -- 172.21.15.33:0/1157766486 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00cd00 con 0x7f7e9c068370 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e90009490 con 0x7f7e9c068370 2024-01-25T16:18:33.453 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e98ff9640 1 -- 172.21.15.33:0/1157766486 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c00ce60 con 0x7f7e9c068370 2024-01-25T16:18:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7e98ff9640 1 -- 172.21.15.33:0/1157766486 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c010430 con 0x7f7e9c068370 2024-01-25T16:18:33.454 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.451+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1a5e80 con 0x7f7e9c068370 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.453+0000 7f7e98ff9640 1 -- 172.21.15.33:0/1157766486 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7e8c0035f0 con 0x7f7e9c068370 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 Processor -- start 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 -- start start 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e98ff9640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 0x7f7e840a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d432240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d434b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d398070 con 0x55fb8d432780 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d395760 con 0x55fb8d429090 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d391050 con 0x55fb8d4299b0 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 0x7f7e840a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:47474/0 (socket says 172.21.15.33:47474) 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/2741459728 learned_addr learned my addr 172.21.15.33:0/2741459728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e98ff9640 1 -- 172.21.15.33:0/1157766486 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e8c014070 con 0x7f7e9c068370 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d434b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9affd640 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d432240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/2741459728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 msgr2=0x55fb8d432240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d432240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/2741459728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 msgr2=0x55fb8d434b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d434b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.454+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/2741459728 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb8d388e70 con 0x55fb8d429090 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0caa00 tx=0x7f7e8c01ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eec08630085520e9 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 0x7f7e840a39e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e94005fd0 tx=0x7f7e94009f00 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7e81ffb640 1 -- 172.21.15.33:0/2741459728 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0b0fe0 con 0x55fb8d429090 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7e81ffb640 1 -- 172.21.15.33:0/2741459728 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c0b2b50 con 0x55fb8d429090 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7e81ffb640 1 -- 172.21.15.33:0/2741459728 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c09e1a0 con 0x55fb8d429090 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/2741459728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 msgr2=0x55fb8d429470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0caa00 tx=0x7f7e8c01ce80 comp rx=0 tx=0).stop 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/2741459728 shutdown_connections 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d434b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d432240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/2741459728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d429470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/2741459728 >> 172.21.15.33:0/2741459728 conn(0x55fb8d4239b0 msgr2=0x55fb8d423d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/2741459728 shutdown_connections 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.455+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/2741459728 wait complete. 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 -- start start 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 Processor -- start 2024-01-25T16:18:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d429090 0x55fb8d42f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d430190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d34a1e0 con 0x55fb8d4299b0 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d349e80 con 0x55fb8d432780 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d30f0c0 con 0x55fb8d429090 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7ea4bfe740 10 client.0 _ll_register_callbacks cb 0x55fb8d334ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7e9affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57544/0 (socket says 172.21.15.33:57544) 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d429090 0x55fb8d42f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7e9affd640 1 -- 172.21.15.33:0/1938828224 learned_addr learned my addr 172.21.15.33:0/1938828224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:33.464 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.456+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d430190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 -- 172.21.15.33:0/1938828224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d429090 msgr2=0x55fb8d42f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d429090 0x55fb8d42f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 -- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 msgr2=0x55fb8d430190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d430190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 -- 172.21.15.33:0/1938828224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c1a5e80 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d430190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000c00 tx=0x7f7e90034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d001aadafb088c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9002c7a0 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e90045070 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb8d38a040 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e9002c900 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7ea4bfe740 10 client.16272 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fb8d38f6d0 con 0x55fb8d4299b0 2024-01-25T16:18:33.465 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fb8d38a040 con 0x55fb8d4299b0 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e90050070 con 0x55fb8d4299b0 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.457+0000 7f7ea4bfe740 10 client.16272 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7ea4bfe740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7ea4bfe740 20 client.16272 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d442540 0x55fb8d444960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7ea4bfe740 4 client.16272 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb8d389d60 con 0x55fb8d442540 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d442540 0x55fb8d444960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7e7affd640 10 client.16272 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.458+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d442540 0x55fb8d444960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c3a4f421f6a5fbe server_cookie=79b190e40a7a52fa in_seq=0 out_seq=0 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.459+0000 7f7e7affd640 1 -- 172.21.15.33:0/1938828224 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb8d389d60 con 0x55fb8d442540 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.459+0000 7f7e7affd640 10 client.16272 handle_command_reply: tid=0 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.459+0000 7f7ea4bfe740 1 client.16272 shutdown 2024-01-25T16:18:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.459+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 msgr2=0x55fb8d42fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.459+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f7e90000c00 tx=0x7f7e90034670 comp rx=0 tx=0).stop 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 shutdown_connections 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d442540 0x55fb8d444960 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d430190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d4299b0 0x55fb8d42fc50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1938828224 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d429090 0x55fb8d42f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7e7affd640 0 client.16272 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 >> 172.21.15.33:0/1938828224 conn(0x55fb8d4239b0 msgr2=0x55fb8d425700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 shutdown_connections 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1938828224 wait complete. 2024-01-25T16:18:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.460+0000 7f7ea4bfe740 20 client.16272 trim_cache size 0 max 16384 2024-01-25T16:18:33.475 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 Processor -- start 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 -- start start 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d431e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d42ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d42c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d389d60 con 0x55fb8d432780 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d38a040 con 0x55fb8d429090 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d434f30 con 0x55fb8d4299b0 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d42c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d431e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d42c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57548/0 (socket says 172.21.15.33:57548) 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9bfff640 1 -- 172.21.15.33:0/3408285903 learned_addr learned my addr 172.21.15.33:0/3408285903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:33.476 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9affd640 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d42ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/3408285903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 msgr2=0x55fb8d42ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d42ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/3408285903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 msgr2=0x55fb8d42c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d42c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.474+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/3408285903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb8d42a590 con 0x55fb8d429090 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.475+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d431e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0caa00 tx=0x7f7e8c003a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3599507f72833da2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.475+0000 7f7e817fa640 1 -- 172.21.15.33:0/3408285903 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c09a750 con 0x55fb8d429090 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.475+0000 7f7e817fa640 1 -- 172.21.15.33:0/3408285903 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c0b2970 con 0x55fb8d429090 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.475+0000 7f7e817fa640 1 -- 172.21.15.33:0/3408285903 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0071c0 con 0x55fb8d429090 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/3408285903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 msgr2=0x55fb8d431e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d431e40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0caa00 tx=0x7f7e8c003a00 comp rx=0 tx=0).stop 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/3408285903 shutdown_connections 2024-01-25T16:18:33.477 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d432780 0x55fb8d42c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d42ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/3408285903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d429090 0x55fb8d431e40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/3408285903 >> 172.21.15.33:0/3408285903 conn(0x55fb8d4239b0 msgr2=0x55fb8d42aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/3408285903 shutdown_connections 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/3408285903 wait complete. 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- start start 2024-01-25T16:18:33.478 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 Processor -- start 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d437b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d438070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d438770 con 0x55fb8d429090 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d4388e0 con 0x55fb8d432780 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb8d498a10 con 0x55fb8d4299b0 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7ea4bfe740 10 client.0 _ll_register_callbacks cb 0x7f7e8c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7e9b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57564/0 (socket says 172.21.15.33:57564) 2024-01-25T16:18:33.481 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1770089582 learned_addr learned my addr 172.21.15.33:0/1770089582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.476+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d437b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9bfff640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d438070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1770089582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 msgr2=0x55fb8d437b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d437b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 msgr2=0x55fb8d438070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d438070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 -- 172.21.15.33:0/1770089582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb8d388e70 con 0x55fb8d429090 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e9b7fe640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c09a180 tx=0x7f7e8c09a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a447ce89bb74c5f2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e7affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c09e1a0 con 0x55fb8d429090 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7e8c0cd070 con 0x55fb8d429090 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e8c0b2b50 con 0x55fb8d429090 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c096760 con 0x55fb8d429090 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7ea4bfe740 10 client.16278 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:33.482 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55fb8d32c060 con 0x55fb8d429090 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.477+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7e8c0b2b50 con 0x55fb8d429090 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e8c01f060 con 0x55fb8d429090 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 10 client.16278 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 10 client.16278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 20 client.16278 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d499c90 0x55fb8d49c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 4 client.16278 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb8d090600 con 0x55fb8d499c90 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d499c90 0x55fb8d49c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7e7affd640 10 client.16278 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:33.483 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.478+0000 7f7e9affd640 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d499c90 0x55fb8d49c080 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316071f9a0345491 server_cookie=9597fe3d850b5b44 in_seq=0 out_seq=0 2024-01-25T16:18:33.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:33 smithi033 ceph-mon[31417]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.583+0000 7f7e7affd640 1 -- 172.21.15.33:0/1770089582 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb8d090600 con 0x55fb8d499c90 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.583+0000 7f7e7affd640 10 client.16278 handle_command_reply: tid=0 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.583+0000 7f7ea4bfe740 1 client.16278 shutdown 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 msgr2=0x55fb8d439740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c09a180 tx=0x7f7e8c09a750 comp rx=0 tx=0).stop 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 shutdown_connections 2024-01-25T16:18:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55fb8d499c90 0x55fb8d49c080 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fb8d432780 0x55fb8d438070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7e7affd640 0 client.16278 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55fb8d4299b0 0x55fb8d437b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 --2- 172.21.15.33:0/1770089582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb8d429090 0x55fb8d439740 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 >> 172.21.15.33:0/1770089582 conn(0x55fb8d4239b0 msgr2=0x55fb8d424db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 shutdown_connections 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 1 -- 172.21.15.33:0/1770089582 wait complete. 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.584+0000 7f7ea4bfe740 20 client.16278 trim_cache size 0 max 16384 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:18:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:18:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:18:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 msgr2=0x7f7e840a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 0x7f7e840a39e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7e94005fd0 tx=0x7f7e94009f00 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 msgr2=0x7f7e9c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0fff10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c005fd0 tx=0x7f7e8c00ee50 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 shutdown_connections 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f7e840a1520 0x7f7e840a39e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e9c106de0 0x7f7e9c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e9c068d10 0x7f7e9c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 --2- 172.21.15.33:0/1157766486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7e9c068370 0x7f7e9c0fff10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 >> 172.21.15.33:0/1157766486 conn(0x7f7e9c076a40 msgr2=0x7f7e9c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 shutdown_connections 2024-01-25T16:18:33.587 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:33.585+0000 7f7ea203d640 1 -- 172.21.15.33:0/1157766486 wait complete. 2024-01-25T16:18:33.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:33 smithi152 ceph-mon[36015]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-01-25T16:18:33.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:33 smithi076 ceph-mon[35642]: pgmap v338: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.1 MiB/s wr, 328 op/s 2024-01-25T16:18:33.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 719008 7.95 MB/sec execute 450 sec latency 64.370 ms 2024-01-25T16:18:34.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 721009 7.96 MB/sec execute 451 sec latency 15.018 ms 2024-01-25T16:18:35.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:35 smithi033 ceph-mon[31417]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-01-25T16:18:35.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:35 smithi152 ceph-mon[36015]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-01-25T16:18:35.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:35 smithi076 ceph-mon[35642]: pgmap v339: 97 pgs: 97 active+clean; 118 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 177 op/s 2024-01-25T16:18:35.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 722496 7.95 MB/sec execute 452 sec latency 16.844 ms 2024-01-25T16:18:36.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:36 smithi033 ceph-mon[31417]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 324 op/s 2024-01-25T16:18:36.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:36 smithi152 ceph-mon[36015]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 324 op/s 2024-01-25T16:18:36.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:36 smithi076 ceph-mon[35642]: pgmap v340: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 3.6 MiB/s wr, 324 op/s 2024-01-25T16:18:36.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 724518 7.96 MB/sec execute 453 sec latency 35.961 ms 2024-01-25T16:18:37.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 725383 7.95 MB/sec execute 454 sec latency 153.301 ms 2024-01-25T16:18:38.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:38 smithi033 ceph-mon[31417]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:38.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:38 smithi152 ceph-mon[36015]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:38.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:38 smithi076 ceph-mon[35642]: pgmap v341: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:38.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 727614 7.96 MB/sec execute 455 sec latency 63.147 ms 2024-01-25T16:18:39.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 728860 7.96 MB/sec execute 456 sec latency 17.284 ms 2024-01-25T16:18:40.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 731475 7.98 MB/sec execute 457 sec latency 15.676 ms 2024-01-25T16:18:41.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:41 smithi033 ceph-mon[31417]: pgmap v342: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:41.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:41 smithi152 ceph-mon[36015]: pgmap v342: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:41.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:41 smithi076 ceph-mon[35642]: pgmap v342: 97 pgs: 97 active+clean; 113 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 232 op/s 2024-01-25T16:18:41.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 732567 7.97 MB/sec execute 458 sec latency 20.533 ms 2024-01-25T16:18:42.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 734983 7.98 MB/sec execute 459 sec latency 52.743 ms 2024-01-25T16:18:43.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:43 smithi033 ceph-mon[31417]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 314 op/s 2024-01-25T16:18:43.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:43 smithi152 ceph-mon[36015]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 314 op/s 2024-01-25T16:18:43.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:43 smithi076 ceph-mon[35642]: pgmap v343: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.4 MiB/s wr, 314 op/s 2024-01-25T16:18:43.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 735759 7.97 MB/sec execute 460 sec latency 33.695 ms 2024-01-25T16:18:44.785 INFO:tasks.workunit.client.0.smithi033.stdout: 1 738204 7.98 MB/sec execute 461 sec latency 22.557 ms 2024-01-25T16:18:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:45 smithi033 ceph-mon[31417]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-01-25T16:18:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:45 smithi152 ceph-mon[36015]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-01-25T16:18:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:45 smithi076 ceph-mon[35642]: pgmap v344: 97 pgs: 97 active+clean; 118 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 228 op/s 2024-01-25T16:18:45.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 739184 7.98 MB/sec execute 462 sec latency 24.911 ms 2024-01-25T16:18:46.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:46 smithi033 ceph-mon[31417]: pgmap v345: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-01-25T16:18:46.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:46 smithi152 ceph-mon[36015]: pgmap v345: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-01-25T16:18:46.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:46 smithi076 ceph-mon[35642]: pgmap v345: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.2 MiB/s wr, 315 op/s 2024-01-25T16:18:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 -- 172.21.15.33:0/1849394457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc0fff60 msgr2=0x7f17cc103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 --2- 172.21.15.33:0/1849394457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc0fff60 0x7f17cc103fa0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009930 tx=0x7f17bc02bc30 comp rx=0 tx=0).stop 2024-01-25T16:18:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 -- 172.21.15.33:0/1849394457 shutdown_connections 2024-01-25T16:18:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 --2- 172.21.15.33:0/1849394457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc104680 0x7f17cc10c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 --2- 172.21.15.33:0/1849394457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc0fff60 0x7f17cc103fa0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 --2- 172.21.15.33:0/1849394457 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 -- 172.21.15.33:0/1849394457 >> 172.21.15.33:0/1849394457 conn(0x7f17cc0761b0 msgr2=0x7f17cc0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 -- 172.21.15.33:0/1849394457 shutdown_connections 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.675+0000 7f17d43ec640 1 -- 172.21.15.33:0/1849394457 wait complete. 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 Processor -- start 2024-01-25T16:18:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 -- start start 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc19dca0 0x7f17cc19e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc19e690 con 0x7f17cc104680 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc19e800 con 0x7f17cc19dca0 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d43ec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc19e970 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d2188640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42048/0 (socket says 172.21.15.33:42048) 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.676+0000 7f17d1987640 1 -- 172.21.15.33:0/2128457595 learned_addr learned my addr 172.21.15.33:0/2128457595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2989640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc19dca0 0x7f17cc19e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 -- 172.21.15.33:0/2128457595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc19dca0 msgr2=0x7f17cc19e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc19dca0 0x7f17cc19e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 -- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 msgr2=0x7f17cc199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 -- 172.21.15.33:0/2128457595 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc009560 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d1987640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d2188640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc1993e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17c800ea90 tx=0x7f17c800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17c37fe640 1 -- 172.21.15.33:0/2128457595 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800cc80 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17c37fe640 1 -- 172.21.15.33:0/2128457595 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c800cde0 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc1a5f50 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17c37fe640 1 -- 172.21.15.33:0/2128457595 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800b760 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.677+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc1a6370 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.680 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.680+0000 7f17c37fe640 1 -- 172.21.15.33:0/2128457595 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f17c8016410 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 Processor -- start 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 -- start start 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f3090 0x55a36e8f3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e8fc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8feb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e862070 con 0x55a36e8fc780 2024-01-25T16:18:46.682 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17c37fe640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 0x7f17b80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e85f760 con 0x55a36e8f39b0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e85b050 con 0x55a36e8f3090 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d2188640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e8fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8feb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d2989640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 0x7f17b80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8feb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42064/0 (socket says 172.21.15.33:42064) 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d1987640 1 -- 172.21.15.33:0/3287754004 learned_addr learned my addr 172.21.15.33:0/3287754004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17d2989640 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f3090 0x55a36e8f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.681+0000 7f17c37fe640 1 -- 172.21.15.33:0/2128457595 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f17c80ccd20 con 0x7f17cc0ff5b0 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 -- 172.21.15.33:0/3287754004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f3090 msgr2=0x55a36e8f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f3090 0x55a36e8f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 -- 172.21.15.33:0/3287754004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 msgr2=0x55a36e8feb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8feb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.683 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 -- 172.21.15.33:0/3287754004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a36e852e70 con 0x55a36e8f39b0 2024-01-25T16:18:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2989640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 0x7f17b80a39b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c4009b60 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17d2188640 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e8fc240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17c80ca430 tx=0x7f17c8069cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2351e8cc1fce567f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17c0bfe640 1 -- 172.21.15.33:0/3287754004 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8099b90 con 0x55a36e8f39b0 2024-01-25T16:18:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17c0bfe640 1 -- 172.21.15.33:0/3287754004 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c809dc30 con 0x55a36e8f39b0 2024-01-25T16:18:46.684 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.682+0000 7f17c0bfe640 1 -- 172.21.15.33:0/3287754004 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80ca810 con 0x55a36e8f39b0 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3287754004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 msgr2=0x55a36e8fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e8fc240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f17c80ca430 tx=0x7f17c8069cc0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3287754004 shutdown_connections 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8feb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e8fc240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3287754004 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f3090 0x55a36e8f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.689 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3287754004 >> 172.21.15.33:0/3287754004 conn(0x55a36e8ed9b0 msgr2=0x55a36e8edd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3287754004 shutdown_connections 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3287754004 wait complete. 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 -- start start 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.683+0000 7f17d6fb1740 1 Processor -- start 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f39b0 0x55a36e8f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8fa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e8141e0 con 0x55a36e8fc780 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e813e80 con 0x55a36e8f3090 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e7d90c0 con 0x55a36e8f39b0 2024-01-25T16:18:46.690 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d6fb1740 10 client.0 _ll_register_callbacks cb 0x55a36e7feac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2989640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8fa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d1987640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f39b0 0x55a36e8f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:43996/0 (socket says 172.21.15.33:43996) 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 -- 172.21.15.33:0/1005042261 learned_addr learned my addr 172.21.15.33:0/1005042261 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 -- 172.21.15.33:0/1005042261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f39b0 msgr2=0x55a36e8f9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f39b0 0x55a36e8f9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 -- 172.21.15.33:0/1005042261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 msgr2=0x55a36e8fa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8fa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 -- 172.21.15.33:0/1005042261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc1a6370 con 0x55a36e8f3090 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17d2188640 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17c8096090 tx=0x7f17c8023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc66dc13585d0ab4 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.691 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.684+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8099b90 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mon.2 v2:172.21.15.76:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f17c80b7e40 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a36e854040 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c80b2200 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 10 client.25814 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a36e8596d0 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mon.2 v2:172.21.15.76:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a36e854040 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mon.2 v2:172.21.15.76:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17c8099110 con 0x55a36e8f3090 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 10 client.25814 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:46.692 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 20 client.25814 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e90c220 0x55a36e90e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 4 client.25814 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:18:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a36e853d60 con 0x55a36e90c220 2024-01-25T16:18:46.693 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.685+0000 7f17d1987640 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e90c220 0x55a36e90e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17b5ffb640 10 client.25814 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d1987640 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e90c220 0x55a36e90e610 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f972ed21ebdb37 server_cookie=fe29d91262310db1 in_seq=0 out_seq=0 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17b5ffb640 1 -- 172.21.15.33:0/1005042261 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a36e853d60 con 0x55a36e90c220 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17b5ffb640 10 client.25814 handle_command_reply: tid=0 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 client.25814 shutdown 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 msgr2=0x55a36e8f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f17c8096090 tx=0x7f17c8023000 comp rx=0 tx=0).stop 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 shutdown_connections 2024-01-25T16:18:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e90c220 0x55a36e90e610 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8fc780 0x55a36e8fa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8f39b0 0x55a36e8f9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1005042261 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e8f9730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17b5ffb640 0 client.25814 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.686+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 >> 172.21.15.33:0/1005042261 conn(0x55a36e8ed9b0 msgr2=0x55a36e8ef700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.687+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 shutdown_connections 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.687+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1005042261 wait complete. 2024-01-25T16:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.687+0000 7f17d6fb1740 20 client.25814 trim_cache size 0 max 16384 2024-01-25T16:18:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.701+0000 7f17d6fb1740 1 Processor -- start 2024-01-25T16:18:46.706 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.701+0000 7f17d6fb1740 1 -- start start 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.701+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.701+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e903410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.701+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e903950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e8f45d0 con 0x55a36e8f3090 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e8ee9e0 con 0x55a36e8f39b0 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e90c020 con 0x55a36e8fc780 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42094/0 (socket says 172.21.15.33:42094) 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 -- 172.21.15.33:0/1475158826 learned_addr learned my addr 172.21.15.33:0/1475158826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d1987640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e903410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2989640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e903950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 -- 172.21.15.33:0/1475158826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 msgr2=0x55a36e903950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e903950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 -- 172.21.15.33:0/1475158826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 msgr2=0x55a36e903410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e903410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 -- 172.21.15.33:0/1475158826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a36e7f6060 con 0x55a36e8f3090 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2989640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e903950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17d2188640 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f17c80cd090 tx=0x7f17c8016ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1dd322f7e8a3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17b7fff640 1 -- 172.21.15.33:0/1475158826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80b2200 con 0x55a36e8f3090 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17b7fff640 1 -- 172.21.15.33:0/1475158826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17c80996d0 con 0x55a36e8f3090 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.702+0000 7f17b7fff640 1 -- 172.21.15.33:0/1475158826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80992e0 con 0x55a36e8f3090 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1475158826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 msgr2=0x55a36e8ec8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f17c80cd090 tx=0x7f17c8016ea0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1475158826 shutdown_connections 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e903950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f39b0 0x55a36e903410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/1475158826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f3090 0x55a36e8ec8d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1475158826 >> 172.21.15.33:0/1475158826 conn(0x55a36e8ed9b0 msgr2=0x55a36e8ec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1475158826 shutdown_connections 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- 172.21.15.33:0/1475158826 wait complete. 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.703+0000 7f17d6fb1740 1 -- start start 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 Processor -- start 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e9022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e9626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e9030f0 con 0x55a36e8f39b0 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e962be0 con 0x55a36e8f3090 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a36e962d20 con 0x55a36e8fc780 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d6fb1740 10 client.0 _ll_register_callbacks cb 0x7f17c40098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d2989640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e9626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42108/0 (socket says 172.21.15.33:42108) 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 -- 172.21.15.33:0/3167420337 learned_addr learned my addr 172.21.15.33:0/3167420337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d2188640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e9022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 -- 172.21.15.33:0/3167420337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 msgr2=0x55a36e9626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e9626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 -- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 msgr2=0x55a36e9022e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e9022e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 -- 172.21.15.33:0/3167420337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a36e8596d0 con 0x55a36e8f39b0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d2989640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e9626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17d1987640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009230 tx=0x7f17bc02ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9d32f24a86466f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.704+0000 7f17b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc045040 con 0x55a36e8f39b0 2024-01-25T16:18:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f17bc004050 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f17c809dc30 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17bc004210 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 10 client.16296 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55a36e81ba80 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17bc061080 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f17bc03c040 con 0x55a36e8f39b0 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 10 client.16296 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 10 client.16296 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 20 client.16296 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e9644e0 0x55a36e966900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 4 client.16296 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:18:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a36e55a600 con 0x55a36e9644e0 2024-01-25T16:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.705+0000 7f17d2188640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e9644e0 0x55a36e966900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.706+0000 7f17b5ffb640 10 client.16296 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.706+0000 7f17d2188640 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e9644e0 0x55a36e966900 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550beb65aee89b52 server_cookie=bf893f3119056a78 in_seq=0 out_seq=0 2024-01-25T16:18:46.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 741463 7.98 MB/sec execute 463 sec latency 47.291 ms 2024-01-25T16:18:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.813+0000 7f17b5ffb640 1 -- 172.21.15.33:0/3167420337 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a36e55a600 con 0x55a36e9644e0 2024-01-25T16:18:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17b5ffb640 10 client.16296 handle_command_reply: tid=0 2024-01-25T16:18:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 client.16296 shutdown 2024-01-25T16:18:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 msgr2=0x55a36e902820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009230 tx=0x7f17bc02ca60 comp rx=0 tx=0).stop 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 shutdown_connections 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55a36e9644e0 0x55a36e966900 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55a36e8fc780 0x55a36e9626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17b5ffb640 0 client.16296 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a36e8f39b0 0x55a36e902820 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 --2- 172.21.15.33:0/3167420337 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a36e8f3090 0x55a36e9022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 >> 172.21.15.33:0/3167420337 conn(0x55a36e8ed9b0 msgr2=0x55a36e8eca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 shutdown_connections 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 1 -- 172.21.15.33:0/3167420337 wait complete. 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.814+0000 7f17d6fb1740 20 client.16296 trim_cache size 0 max 16384 2024-01-25T16:18:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:18:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:18:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:18:46.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 msgr2=0x7f17b80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 0x7f17b80a39b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f17c4005fd0 tx=0x7f17c4009b60 comp rx=0 tx=0).stop 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 msgr2=0x7f17cc1993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc1993e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f17c800ea90 tx=0x7f17c800ef50 comp rx=0 tx=0).stop 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 shutdown_connections 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f17b80a14f0 0x7f17b80a39b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17cc19dca0 0x7f17cc19e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17cc104680 0x7f17cc199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 --2- 172.21.15.33:0/2128457595 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f17cc0ff5b0 0x7f17cc1993e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 >> 172.21.15.33:0/2128457595 conn(0x7f17cc0761b0 msgr2=0x7f17cc103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 shutdown_connections 2024-01-25T16:18:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:46.815+0000 7f17d43ec640 1 -- 172.21.15.33:0/2128457595 wait complete. 2024-01-25T16:18:47.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 742730 7.98 MB/sec execute 464 sec latency 63.660 ms 2024-01-25T16:18:48.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:48 smithi033 ceph-mon[31417]: pgmap v346: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:48.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:48 smithi152 ceph-mon[36015]: pgmap v346: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:48.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:48 smithi076 ceph-mon[35642]: pgmap v346: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:48.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 744737 7.98 MB/sec execute 465 sec latency 82.691 ms 2024-01-25T16:18:49.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 746119 7.99 MB/sec execute 466 sec latency 41.971 ms 2024-01-25T16:18:50.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 748162 7.99 MB/sec execute 467 sec latency 17.093 ms 2024-01-25T16:18:51.564 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:51 smithi033 ceph-mon[31417]: pgmap v347: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:51.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:51 smithi152 ceph-mon[36015]: pgmap v347: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:51.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:51 smithi076 ceph-mon[35642]: pgmap v347: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 168 op/s 2024-01-25T16:18:51.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 749541 8.00 MB/sec execute 468 sec latency 90.426 ms 2024-01-25T16:18:52.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 751515 8.00 MB/sec execute 469 sec latency 16.999 ms 2024-01-25T16:18:53.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:53 smithi033 ceph-mon[31417]: pgmap v348: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-01-25T16:18:53.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:53 smithi152 ceph-mon[36015]: pgmap v348: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-01-25T16:18:53.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:53 smithi076 ceph-mon[35642]: pgmap v348: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 316 op/s 2024-01-25T16:18:53.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 753007 8.00 MB/sec execute 470 sec latency 102.096 ms 2024-01-25T16:18:54.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 754630 8.00 MB/sec execute 471 sec latency 20.539 ms 2024-01-25T16:18:55.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:55 smithi033 ceph-mon[31417]: pgmap v349: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:18:55.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:55 smithi152 ceph-mon[36015]: pgmap v349: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:18:55.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:55 smithi076 ceph-mon[35642]: pgmap v349: 97 pgs: 97 active+clean; 115 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:18:55.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 756562 8.01 MB/sec execute 472 sec latency 13.931 ms 2024-01-25T16:18:56.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:56 smithi033 ceph-mon[31417]: pgmap v350: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-01-25T16:18:56.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:56 smithi152 ceph-mon[36015]: pgmap v350: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-01-25T16:18:56.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:56 smithi076 ceph-mon[35642]: pgmap v350: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.7 MiB/s wr, 317 op/s 2024-01-25T16:18:56.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 758232 8.01 MB/sec execute 473 sec latency 23.825 ms 2024-01-25T16:18:57.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 760112 8.02 MB/sec execute 474 sec latency 24.061 ms 2024-01-25T16:18:58.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:18:58 smithi033 ceph-mon[31417]: pgmap v351: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:18:58.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:18:58 smithi152 ceph-mon[36015]: pgmap v351: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:18:58.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:18:58 smithi076 ceph-mon[35642]: pgmap v351: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:18:58.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 761561 8.01 MB/sec execute 475 sec latency 79.477 ms 2024-01-25T16:18:59.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 763472 8.02 MB/sec execute 476 sec latency 198.957 ms 2024-01-25T16:18:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 -- 172.21.15.33:0/81041765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8340688b0 msgr2=0x7fa834068cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 --2- 172.21.15.33:0/81041765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8340688b0 0x7fa834068cf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009900 tx=0x7fa81c02bcd0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 -- 172.21.15.33:0/81041765 shutdown_connections 2024-01-25T16:18:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 --2- 172.21.15.33:0/81041765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa83410d4b0 0x7fa83410f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 --2- 172.21.15.33:0/81041765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8340688b0 0x7fa834068cf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 --2- 172.21.15.33:0/81041765 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa834101c70 0x7fa834068370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 -- 172.21.15.33:0/81041765 >> 172.21.15.33:0/81041765 conn(0x7fa834076a40 msgr2=0x7fa834076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 -- 172.21.15.33:0/81041765 shutdown_connections 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.904+0000 7fa83911d640 1 -- 172.21.15.33:0/81041765 wait complete. 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 Processor -- start 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 -- start start 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa83410d4b0 0x7fa83419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83419e230 con 0x7fa834101c70 2024-01-25T16:18:59.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83419e3a0 con 0x7fa8340688b0 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83911d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83419e560 con 0x7fa83410d4b0 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa832d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa832d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:36846/0 (socket says 172.21.15.33:36846) 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49830/0 (socket says 172.21.15.33:49830) 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa832d9d640 1 -- 172.21.15.33:0/854754210 learned_addr learned my addr 172.21.15.33:0/854754210 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.905+0000 7fa83359e640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa83410d4b0 0x7fa83419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 -- 172.21.15.33:0/854754210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa83410d4b0 msgr2=0x7fa83419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa83410d4b0 0x7fa83419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 -- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 msgr2=0x7fa8341998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.908 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 -- 172.21.15.33:0/854754210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009840 con 0x7fa8340688b0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa82bfff640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa832d9d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa82400e9f0 tx=0x7fa82400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa830d99640 1 -- 172.21.15.33:0/854754210 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824009720 con 0x7fa8340688b0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c009560 con 0x7fa8340688b0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa830d99640 1 -- 172.21.15.33:0/854754210 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82400cd80 con 0x7fa8340688b0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa830d99640 1 -- 172.21.15.33:0/854754210 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824010430 con 0x7fa8340688b0 2024-01-25T16:18:59.909 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.906+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8341a5ee0 con 0x7fa8340688b0 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.909+0000 7fa83bdba740 1 Processor -- start 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.909+0000 7fa83bdba740 1 -- start start 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x559100295470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591002959b0 0x55910029e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100204070 con 0x55910029e780 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100201760 con 0x5591002959b0 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83bdba740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591001fd050 con 0x559100295090 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa830d99640 1 -- 172.21.15.33:0/854754210 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fa824005020 con 0x7fa8340688b0 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.916 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x559100295470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49838/0 (socket says 172.21.15.33:49838) 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa82bfff640 1 -- 172.21.15.33:0/1228929756 learned_addr learned my addr 172.21.15.33:0/1228929756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa832d9d640 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591002959b0 0x55910029e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 -- 172.21.15.33:0/1228929756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591002959b0 msgr2=0x55910029e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591002959b0 0x55910029e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 -- 172.21.15.33:0/1228929756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 msgr2=0x5591002a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 -- 172.21.15.33:0/1228929756 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591001f4e70 con 0x559100295090 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa82bfff640 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa83359e640 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x559100295470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa82c011540 tx=0x7fa82c011a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b40dae110d60e8 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa830d99640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 0x7fa8200a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.910+0000 7fa8297fa640 1 -- 172.21.15.33:0/1228929756 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004d20 con 0x559100295090 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa832d9d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 0x7fa8200a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.917 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa8297fa640 1 -- 172.21.15.33:0/1228929756 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa82c004e80 con 0x559100295090 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa8297fa640 1 -- 172.21.15.33:0/1228929756 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00f820 con 0x559100295090 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa830d99640 1 -- 172.21.15.33:0/854754210 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa824014070 con 0x7fa8340688b0 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 -- 172.21.15.33:0/1228929756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 msgr2=0x559100295470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa832d9d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 0x7fa8200a39b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8240caa80 tx=0x7fa824004a20 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x559100295470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa82c011540 tx=0x7fa82c011a00 comp rx=0 tx=0).stop 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 -- 172.21.15.33:0/1228929756 shutdown_connections 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55910029e780 0x5591002a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591002959b0 0x55910029e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1228929756 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x559100295470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 -- 172.21.15.33:0/1228929756 >> 172.21.15.33:0/1228929756 conn(0x55910028f9b0 msgr2=0x55910028fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 -- 172.21.15.33:0/1228929756 shutdown_connections 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.911+0000 7fa83bdba740 1 -- 172.21.15.33:0/1228929756 wait complete. 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 -- start start 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 Processor -- start 2024-01-25T16:18:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x55910029c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591001b61e0 con 0x559100295090 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591001b5e80 con 0x55910029e780 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55910017b0c0 con 0x5591002959b0 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83bdba740 10 client.0 _ll_register_callbacks cb 0x5591001a0ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49852/0 (socket says 172.21.15.33:49852) 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:37976/0 (socket says 172.21.15.33:37976) 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 -- 172.21.15.33:0/3610758284 learned_addr learned my addr 172.21.15.33:0/3610758284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x55910029c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 -- 172.21.15.33:0/3610758284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 msgr2=0x55910029bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 -- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 msgr2=0x55910029c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x55910029c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa83359e640 1 -- 172.21.15.33:0/3610758284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c009d60 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.912+0000 7fa82bfff640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83359e640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa82c004720 tx=0x7fa82c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae8316098665a13 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004e80 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa82c0041d0 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591001f5d60 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa82c00f5c0 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 10 client.16308 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5591001fb6d0 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591001f5d60 con 0x559100295090 2024-01-25T16:18:59.920 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa82c005340 con 0x559100295090 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 10 client.16308 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 20 client.16308 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591002ae490 0x5591002b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 4 client.16308 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.913+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590ffefc600 con 0x5591002ae490 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.914+0000 7fa82bfff640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591002ae490 0x5591002b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.914+0000 7fa82bfff640 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591002ae490 0x5591002b0880 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e38264b0fd3b69ab server_cookie=e1c6ce98ff33bcf4 in_seq=0 out_seq=0 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.914+0000 7fa81a7fc640 10 client.16308 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.915+0000 7fa81a7fc640 1 -- 172.21.15.33:0/3610758284 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590ffefc600 con 0x5591002ae490 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.915+0000 7fa81a7fc640 10 client.16308 handle_command_reply: tid=0 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.915+0000 7fa83bdba740 1 client.16308 shutdown 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 msgr2=0x55910029b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa82c004720 tx=0x7fa82c004620 comp rx=0 tx=0).stop 2024-01-25T16:18:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 shutdown_connections 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591002ae490 0x5591002b0880 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x55910029c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5591002959b0 0x55910029bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 --2- 172.21.15.33:0/3610758284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559100295090 0x55910029b700 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 >> 172.21.15.33:0/3610758284 conn(0x55910028f9b0 msgr2=0x559100291700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa81a7fc640 0 client.16308 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 shutdown_connections 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 1 -- 172.21.15.33:0/3610758284 wait complete. 2024-01-25T16:18:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.916+0000 7fa83bdba740 20 client.16308 trim_cache size 0 max 16384 2024-01-25T16:18:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 Processor -- start 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 -- start start 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x55910029ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x559100299ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591001f5d60 con 0x5591002959b0 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55910029cb60 con 0x55910029e780 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.930+0000 7fa83bdba740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55910029c6d0 con 0x559100295090 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49868/0 (socket says 172.21.15.33:49868) 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 -- 172.21.15.33:0/512275246 learned_addr learned my addr 172.21.15.33:0/512275246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa83359e640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x55910029ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa83359e640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x559100299ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 -- 172.21.15.33:0/512275246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 msgr2=0x55910029ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x55910029ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 -- 172.21.15.33:0/512275246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 msgr2=0x559100299ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x559100299ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 -- 172.21.15.33:0/512275246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559100198060 con 0x5591002959b0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa83359e640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x559100299ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa82bfff640 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa81c036040 tx=0x7fa81c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519d8c4736543bd6 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa828ff9640 1 -- 172.21.15.33:0/512275246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c047070 con 0x5591002959b0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa828ff9640 1 -- 172.21.15.33:0/512275246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c004510 con 0x5591002959b0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.931+0000 7fa828ff9640 1 -- 172.21.15.33:0/512275246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c0483d0 con 0x5591002959b0 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- 172.21.15.33:0/512275246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 msgr2=0x5591002b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fa81c036040 tx=0x7fa81c02e040 comp rx=0 tx=0).stop 2024-01-25T16:18:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- 172.21.15.33:0/512275246 shutdown_connections 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55910029e780 0x559100299ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002b51b0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 --2- 172.21.15.33:0/512275246 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559100295090 0x55910029ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- 172.21.15.33:0/512275246 >> 172.21.15.33:0/512275246 conn(0x55910028f9b0 msgr2=0x55910029ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- 172.21.15.33:0/512275246 shutdown_connections 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- 172.21.15.33:0/512275246 wait complete. 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 -- start start 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.932+0000 7fa83bdba740 1 Processor -- start 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 0x5591002a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55910029e780 0x559100303930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100303e70 con 0x5591002959b0 2024-01-25T16:18:59.937 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559100303fb0 con 0x559100295090 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591003040f0 con 0x55910029e780 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 0x5591002a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 10 client.0 _ll_register_callbacks cb 0x7fa82c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49884/0 (socket says 172.21.15.33:49884) 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 0x5591002a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:36890/0 (socket says 172.21.15.33:36890) 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 -- 172.21.15.33:0/1038309894 learned_addr learned my addr 172.21.15.33:0/1038309894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83359e640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55910029e780 0x559100303930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 -- 172.21.15.33:0/1038309894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55910029e780 msgr2=0x559100303930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55910029e780 0x559100303930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 -- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 msgr2=0x5591002a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 0x5591002a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:18:59.938 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 -- 172.21.15.33:0/1038309894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c030070 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa82bfff640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02bd10 tx=0x7fa81c048140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca162f45c4d2afa server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa81a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c055070 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa81c02caf0 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.933+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591001f6040 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c047030 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 10 client.16314 fetch_fsmap learned FSMap version 37 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5591001bda80 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591001f6040 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa81c058030 con 0x5591002959b0 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 10 client.16314 fetch_fsmap finished waiting for FSMap version 37 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 10 client.16314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 20 client.16314 populate_metadata read hostname 'smithi033' 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591003055f0 0x5591003079e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:18:59.939 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 4 client.16314 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:18:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5590fff4fc50 con 0x5591003055f0 2024-01-25T16:18:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83359e640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591003055f0 0x5591003079e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:18:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa81a7fc640 10 client.16314 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:18:59.940 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:18:59.934+0000 7fa83359e640 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591003055f0 0x5591003079e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a1026404c63185 server_cookie=423e386d7cdda2b3 in_seq=0 out_seq=0 2024-01-25T16:19:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.041+0000 7fa81a7fc640 1 -- 172.21.15.33:0/1038309894 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5590fff4fc50 con 0x5591003055f0 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.041+0000 7fa81a7fc640 10 client.16314 handle_command_reply: tid=0 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.041+0000 7fa83bdba740 1 client.16314 shutdown 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.041+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 msgr2=0x5591002a3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.041+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa81c02bd10 tx=0x7fa81c048140 comp rx=0 tx=0).stop 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 shutdown_connections 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x5591003055f0 0x5591003079e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55910029e780 0x559100303930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591002959b0 0x5591002a3b80 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 --2- 172.21.15.33:0/1038309894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559100295090 0x5591002a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 >> 172.21.15.33:0/1038309894 conn(0x55910028f9b0 msgr2=0x5591002b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa81a7fc640 0 client.16314 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:00.042 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 shutdown_connections 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 1 -- 172.21.15.33:0/1038309894 wait complete. 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83bdba740 20 client.16314 trim_cache size 0 max 16384 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:19:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.042+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 msgr2=0x7fa8200a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 0x7fa8200a39b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa8240caa80 tx=0x7fa824004a20 comp rx=0 tx=0).stop 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 msgr2=0x7fa834199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa82400e9f0 tx=0x7fa82400edf0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 shutdown_connections 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8200a14f0 0x7fa8200a39b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa83410d4b0 0x7fa83419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa834101c70 0x7fa8341998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 --2- 172.21.15.33:0/854754210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8340688b0 0x7fa834199370 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 >> 172.21.15.33:0/854754210 conn(0x7fa834076a40 msgr2=0x7fa8340fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 shutdown_connections 2024-01-25T16:19:00.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:00.043+0000 7fa83911d640 1 -- 172.21.15.33:0/854754210 wait complete. 2024-01-25T16:19:00.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 764283 8.01 MB/sec execute 477 sec latency 87.075 ms 2024-01-25T16:19:01.129 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:19:01.130 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:19:01.130 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:00 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/2643818713 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d62d48b520 con 0x55d62d461000 2024-01-25T16:19:01.130 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:19:01.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:01 smithi033 ceph-mon[31417]: pgmap v352: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:19:01.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:01 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:19:01.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:01 smithi152 ceph-mon[36015]: pgmap v352: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:19:01.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:01 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:19:01.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:01 smithi076 ceph-mon[35642]: pgmap v352: 97 pgs: 97 active+clean; 121 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 230 op/s 2024-01-25T16:19:01.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:01 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:19:01.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 765818 8.01 MB/sec execute 478 sec latency 23.771 ms 2024-01-25T16:19:02.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:02 smithi033 ceph-mon[31417]: osdmap e104: 12 total, 12 up, 12 in 2024-01-25T16:19:02.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:02 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/1287786698,v1:172.21.15.152:6837/1287786698] up:boot 2024-01-25T16:19:02.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:02 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:02.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:02 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:02.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:02 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:02.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:02 smithi152 ceph-mon[36015]: osdmap e104: 12 total, 12 up, 12 in 2024-01-25T16:19:02.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:02 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/1287786698,v1:172.21.15.152:6837/1287786698] up:boot 2024-01-25T16:19:02.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:02 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:02.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:02 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:02.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:02 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:02.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:02 smithi076 ceph-mon[35642]: osdmap e104: 12 total, 12 up, 12 in 2024-01-25T16:19:02.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:02 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/1287786698,v1:172.21.15.152:6837/1287786698] up:boot 2024-01-25T16:19:02.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:02 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:02.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:02 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:02.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:02 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:02.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 767069 8.01 MB/sec execute 479 sec latency 59.903 ms 2024-01-25T16:19:03.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:03 smithi033 ceph-mon[31417]: pgmap v354: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-01-25T16:19:03.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:03 smithi033 ceph-mon[31417]: osdmap e105: 12 total, 12 up, 12 in 2024-01-25T16:19:03.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:03 smithi152 ceph-mon[36015]: pgmap v354: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-01-25T16:19:03.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:03 smithi152 ceph-mon[36015]: osdmap e105: 12 total, 12 up, 12 in 2024-01-25T16:19:03.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:03 smithi076 ceph-mon[35642]: pgmap v354: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-01-25T16:19:03.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:03 smithi076 ceph-mon[35642]: osdmap e105: 12 total, 12 up, 12 in 2024-01-25T16:19:03.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 767482 8.00 MB/sec execute 480 sec latency 71.184 ms 2024-01-25T16:19:04.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:04 smithi033 ceph-mon[31417]: pgmap v356: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-01-25T16:19:04.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:04 smithi152 ceph-mon[36015]: pgmap v356: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-01-25T16:19:04.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:04 smithi076 ceph-mon[35642]: pgmap v356: 97 pgs: 97 active+clean; 110 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 275 op/s 2024-01-25T16:19:04.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 769375 8.00 MB/sec execute 481 sec latency 34.816 ms 2024-01-25T16:19:05.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 770869 8.00 MB/sec execute 482 sec latency 15.753 ms 2024-01-25T16:19:06.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 772854 8.00 MB/sec execute 483 sec latency 18.984 ms 2024-01-25T16:19:07.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:07 smithi033 ceph-mon[31417]: pgmap v357: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:07.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:07 smithi152 ceph-mon[36015]: pgmap v357: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:07.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:07 smithi076 ceph-mon[35642]: pgmap v357: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:07.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 774328 8.01 MB/sec execute 484 sec latency 15.898 ms 2024-01-25T16:19:08.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 775964 8.01 MB/sec execute 485 sec latency 16.036 ms 2024-01-25T16:19:09.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:09 smithi033 ceph-mon[31417]: pgmap v358: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:09.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:09 smithi152 ceph-mon[36015]: pgmap v358: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:09.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:09 smithi076 ceph-mon[35642]: pgmap v358: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 387 op/s 2024-01-25T16:19:09.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 777713 8.01 MB/sec execute 486 sec latency 81.669 ms 2024-01-25T16:19:10.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 779199 8.01 MB/sec execute 487 sec latency 24.334 ms 2024-01-25T16:19:11.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:11 smithi033 ceph-mon[31417]: pgmap v359: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-01-25T16:19:11.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:11 smithi033 ceph-mon[31417]: osdmap e106: 12 total, 12 up, 12 in 2024-01-25T16:19:11.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:11 smithi152 ceph-mon[36015]: pgmap v359: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-01-25T16:19:11.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:11 smithi152 ceph-mon[36015]: osdmap e106: 12 total, 12 up, 12 in 2024-01-25T16:19:11.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:11 smithi076 ceph-mon[35642]: pgmap v359: 97 pgs: 97 active+clean; 112 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 99 op/s 2024-01-25T16:19:11.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:11 smithi076 ceph-mon[35642]: osdmap e106: 12 total, 12 up, 12 in 2024-01-25T16:19:11.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 780966 8.01 MB/sec execute 488 sec latency 22.255 ms 2024-01-25T16:19:12.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 782113 8.01 MB/sec execute 489 sec latency 20.349 ms 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 -- 172.21.15.33:0/1999830657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c104b90 msgr2=0x7fa11c104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 --2- 172.21.15.33:0/1999830657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c104b90 0x7fa11c104f70 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa10c009900 tx=0x7fa10c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 -- 172.21.15.33:0/1999830657 shutdown_connections 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 --2- 172.21.15.33:0/1999830657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 --2- 172.21.15.33:0/1999830657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c0ffe40 0x7fa11c1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 --2- 172.21.15.33:0/1999830657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c104b90 0x7fa11c104f70 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 -- 172.21.15.33:0/1999830657 >> 172.21.15.33:0/1999830657 conn(0x7fa11c076a40 msgr2=0x7fa11c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 -- 172.21.15.33:0/1999830657 shutdown_connections 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.150+0000 7fa12126e640 1 -- 172.21.15.33:0/1999830657 wait complete. 2024-01-25T16:19:13.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 Processor -- start 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 -- start start 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c1977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c104b90 0x7fa11c19bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19c120 con 0x7fa11c0ffe40 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19c290 con 0x7fa11c100800 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa12126e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c19c450 con 0x7fa11c104b90 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa11affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa11affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57070/0 (socket says 172.21.15.33:57070) 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa11affd640 1 -- 172.21.15.33:0/2351753999 learned_addr learned my addr 172.21.15.33:0/2351753999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa1127fc640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c1977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.151+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c104b90 0x7fa11c19bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 -- 172.21.15.33:0/2351753999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c104b90 msgr2=0x7fa11c19bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c104b90 0x7fa11c19bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 -- 172.21.15.33:0/2351753999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 msgr2=0x7fa11c1977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c1977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 -- 172.21.15.33:0/2351753999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c009560 con 0x7fa11c0ffe40 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa1127fc640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa11affd640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005940 tx=0x7fa10c02c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa118ff9640 1 -- 172.21.15.33:0/2351753999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c004250 con 0x7fa11c0ffe40 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c19c6d0 con 0x7fa11c0ffe40 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa118ff9640 1 -- 172.21.15.33:0/2351753999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c0043b0 con 0x7fa11c0ffe40 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa118ff9640 1 -- 172.21.15.33:0/2351753999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c003b90 con 0x7fa11c0ffe40 2024-01-25T16:19:13.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.152+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a3fb0 con 0x7fa11c0ffe40 2024-01-25T16:19:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.154+0000 7fa118ff9640 1 -- 172.21.15.33:0/2351753999 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fa10c003cf0 con 0x7fa11c0ffe40 2024-01-25T16:19:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 Processor -- start 2024-01-25T16:19:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 -- start start 2024-01-25T16:19:13.161 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa118ff9640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 0x7fa1080a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c55650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c57f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0bb7070 con 0x55f0b0c55b90 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0bb4760 con 0x55f0b0c47bb0 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa123e34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0bb0050 con 0x55f0b0c484d0 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 0x7fa1080a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa118ff9640 1 -- 172.21.15.33:0/2351753999 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa10c02de40 con 0x7fa11c0ffe40 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:54658/0 (socket says 172.21.15.33:54658) 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 -- 172.21.15.33:0/4172074705 learned_addr learned my addr 172.21.15.33:0/4172074705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:13.162 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa1127fc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c55650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c57f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 -- 172.21.15.33:0/4172074705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 msgr2=0x55f0b0c55650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c55650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 -- 172.21.15.33:0/4172074705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 msgr2=0x55f0b0c57f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c57f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa11affd640 1 -- 172.21.15.33:0/4172074705 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b0ba7e70 con 0x55f0b0c47bb0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.155+0000 7fa1127fc640 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c55650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.156+0000 7fa11affd640 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c4320 tx=0x7fa10c0bc000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6294187c478515b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.156+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 0x7fa1080a39b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114009e50 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.156+0000 7fa1117fa640 1 -- 172.21.15.33:0/4172074705 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0bc4c0 con 0x55f0b0c47bb0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.156+0000 7fa1117fa640 1 -- 172.21.15.33:0/4172074705 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c0c0610 con 0x55f0b0c47bb0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.156+0000 7fa1117fa640 1 -- 172.21.15.33:0/4172074705 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0ecf80 con 0x55f0b0c47bb0 2024-01-25T16:19:13.163 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- 172.21.15.33:0/4172074705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 msgr2=0x55f0b0c47f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0c4320 tx=0x7fa10c0bc000 comp rx=0 tx=0).stop 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- 172.21.15.33:0/4172074705 shutdown_connections 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c57f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c55650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 --2- 172.21.15.33:0/4172074705 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c47f90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- 172.21.15.33:0/4172074705 >> 172.21.15.33:0/4172074705 conn(0x55f0b0c429b0 msgr2=0x55f0b0c42d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- 172.21.15.33:0/4172074705 shutdown_connections 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- 172.21.15.33:0/4172074705 wait complete. 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.157+0000 7fa123e34740 1 -- start start 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 Processor -- start 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.164 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0b691e0 con 0x55f0b0c55b90 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0b68e80 con 0x55f0b0c47bb0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0b2e0c0 con 0x55f0b0c484d0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa123e34740 10 client.0 _ll_register_callbacks cb 0x55f0b0b53ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57088/0 (socket says 172.21.15.33:57088) 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 -- 172.21.15.33:0/1865158919 learned_addr learned my addr 172.21.15.33:0/1865158919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11affd640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c55630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa1127fc640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 -- 172.21.15.33:0/1865158919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 msgr2=0x55f0b0c52a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c52a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 -- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 msgr2=0x55f0b0c55630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c55630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.165 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 -- 172.21.15.33:0/1865158919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a3fb0 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11affd640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa114009ab0 tx=0x7fa114004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabc3e1b2e348120 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.158+0000 7fa1027fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa114012ca0 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa114012e00 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0b0ba9040 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa114021410 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 10 client.16332 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f0b0bae6d0 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa114035020 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1140294f0 con 0x55f0b0c55b90 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 10 client.16332 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 20 client.16332 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:13.166 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0c656e0 0x55f0b0c67b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 4 client.16332 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0b0ba8d60 con 0x55f0b0c656e0 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.159+0000 7fa11affd640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0c656e0 0x55f0b0c67b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.160+0000 7fa1027fc640 10 client.16332 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.160+0000 7fa11affd640 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0c656e0 0x55f0b0c67b00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d7da717d2d60207 server_cookie=8e6af93b6877db54 in_seq=0 out_seq=0 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa1027fc640 1 -- 172.21.15.33:0/1865158919 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0b0ba8d60 con 0x55f0b0c656e0 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa1027fc640 10 client.16332 handle_command_reply: tid=0 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 client.16332 shutdown 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 msgr2=0x55f0b0c53120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fa114009ab0 tx=0x7fa114004ab0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 shutdown_connections 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0c656e0 0x55f0b0c67b00 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c55b90 0x55f0b0c53120 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.167 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 --2- 172.21.15.33:0/1865158919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa1027fc640 0 client.16332 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 >> 172.21.15.33:0/1865158919 conn(0x55f0b0c429b0 msgr2=0x55f0b0c44700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 shutdown_connections 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 1 -- 172.21.15.33:0/1865158919 wait complete. 2024-01-25T16:19:13.168 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.161+0000 7fa123e34740 20 client.16332 trim_cache size 0 max 16384 2024-01-25T16:19:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 Processor -- start 2024-01-25T16:19:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 -- start start 2024-01-25T16:19:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.180 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c55b90 0x55f0b0c51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c62100 con 0x55f0b0c47bb0 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c53ec0 con 0x55f0b0c55b90 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa123e34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c427e0 con 0x55f0b0c484d0 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa11affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa11affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57090/0 (socket says 172.21.15.33:57090) 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa11affd640 1 -- 172.21.15.33:0/387254784 learned_addr learned my addr 172.21.15.33:0/387254784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa1127fc640 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.176+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c55b90 0x55f0b0c51300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 -- 172.21.15.33:0/387254784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 msgr2=0x55f0b0c5cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 -- 172.21.15.33:0/387254784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c55b90 msgr2=0x55f0b0c51300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.181 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c55b90 0x55f0b0c51300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 -- 172.21.15.33:0/387254784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0b0b4b060 con 0x55f0b0c47bb0 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa11affd640 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0b89c0 tx=0x7fa10c0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110f8027dd0fd6b7 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa110ff9640 1 -- 172.21.15.33:0/387254784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0bbe20 con 0x55f0b0c47bb0 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa110ff9640 1 -- 172.21.15.33:0/387254784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c0bc4c0 con 0x55f0b0c47bb0 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.177+0000 7fa110ff9640 1 -- 172.21.15.33:0/387254784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10c0bb9b0 con 0x55f0b0c47bb0 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- 172.21.15.33:0/387254784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 msgr2=0x55f0b0c5c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa10c0b89c0 tx=0x7fa10c0dd040 comp rx=0 tx=0).stop 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- 172.21.15.33:0/387254784 shutdown_connections 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c55b90 0x55f0b0c51300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- 172.21.15.33:0/387254784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c5d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- 172.21.15.33:0/387254784 >> 172.21.15.33:0/387254784 conn(0x55f0b0c429b0 msgr2=0x55f0b0c61080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- 172.21.15.33:0/387254784 shutdown_connections 2024-01-25T16:19:13.182 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- 172.21.15.33:0/387254784 wait complete. 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- start start 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 Processor -- start 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c55b90 0x55f0b0c5b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c5cf10 con 0x55f0b0c484d0 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c5b5a0 con 0x55f0b0c47bb0 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.178+0000 7fa123e34740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0b0c5b710 con 0x55f0b0c55b90 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa123e34740 10 client.0 _ll_register_callbacks cb 0x7fa10c004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57106/0 (socket says 172.21.15.33:57106) 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 -- 172.21.15.33:0/2271740045 learned_addr learned my addr 172.21.15.33:0/2271740045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa11affd640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.183 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa11b7fe640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c55b90 0x55f0b0c5b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 -- 172.21.15.33:0/2271740045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c55b90 msgr2=0x55f0b0c5b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c55b90 0x55f0b0c5b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 -- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 msgr2=0x55f0b0c5c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 -- 172.21.15.33:0/2271740045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0b0ba7e70 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1127fc640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa104012c80 tx=0x7fa10400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee1c0f272ff4585 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1027fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104017040 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa10c0c0610 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa104004250 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa104004410 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa123e34740 10 client.16338 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.179+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x55f0b0b70a80 con 0x55f0b0c484d0 2024-01-25T16:19:13.184 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa10c0c0610 con 0x55f0b0c484d0 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa1040104c0 con 0x55f0b0c484d0 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 10 client.16338 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 20 client.16338 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0cbd050 0x55f0b0cbf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 4 client.16338 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0b08af600 con 0x55f0b0cbd050 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa11affd640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0cbd050 0x55f0b0cbf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa11affd640 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0cbd050 0x55f0b0cbf470 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b205d42a748481 server_cookie=cfc93292050cb9d4 in_seq=0 out_seq=0 2024-01-25T16:19:13.185 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.180+0000 7fa1027fc640 10 client.16338 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa1027fc640 1 -- 172.21.15.33:0/2271740045 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0b08af600 con 0x55f0b0cbd050 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa1027fc640 10 client.16338 handle_command_reply: tid=0 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 client.16338 shutdown 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 msgr2=0x55f0b0c5ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa104012c80 tx=0x7fa10400f790 comp rx=0 tx=0).stop 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 shutdown_connections 2024-01-25T16:19:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55f0b0cbd050 0x55f0b0cbf470 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55f0b0c55b90 0x55f0b0c5b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa1027fc640 0 client.16338 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f0b0c484d0 0x55f0b0c5ab20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 --2- 172.21.15.33:0/2271740045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f0b0c47bb0 0x55f0b0c5c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 >> 172.21.15.33:0/2271740045 conn(0x55f0b0c429b0 msgr2=0x55f0b0c61080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 shutdown_connections 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 1 -- 172.21.15.33:0/2271740045 wait complete. 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.309+0000 7fa123e34740 20 client.16338 trim_cache size 0 max 16384 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:19:13.310 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:19:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:19:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 msgr2=0x7fa1080a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 0x7fa1080a39b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa114005fd0 tx=0x7fa114009e50 comp rx=0 tx=0).stop 2024-01-25T16:19:13.311 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 msgr2=0x7fa11c197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa10c005940 tx=0x7fa10c02c670 comp rx=0 tx=0).stop 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 shutdown_connections 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa1080a14f0 0x7fa1080a39b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa11c104b90 0x7fa11c19bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa11c100800 0x7fa11c1977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 --2- 172.21.15.33:0/2351753999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa11c0ffe40 0x7fa11c197260 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 >> 172.21.15.33:0/2351753999 conn(0x7fa11c076a40 msgr2=0x7fa11c068650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 shutdown_connections 2024-01-25T16:19:13.312 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:13.310+0000 7fa12126e640 1 -- 172.21.15.33:0/2351753999 wait complete. 2024-01-25T16:19:13.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:13 smithi033 ceph-mon[31417]: pgmap v361: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:13.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:19:13.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:13 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:13 smithi152 ceph-mon[36015]: pgmap v361: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:19:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:13 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:13 smithi076 ceph-mon[35642]: pgmap v361: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:13.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:19:13.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:13 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:13.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 784109 8.02 MB/sec execute 490 sec latency 110.492 ms 2024-01-25T16:19:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:14 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:19:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:14 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:19:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:14 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:14.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:14 smithi033 ceph-mon[31417]: pgmap v362: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:14.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:14 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:19:14.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:14 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:19:14.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:14 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:14.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:14 smithi152 ceph-mon[36015]: pgmap v362: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:14.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:14 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:19:14.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:14 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:19:14.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:14 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:19:14.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:14 smithi076 ceph-mon[35642]: pgmap v362: 97 pgs: 97 active+clean; 119 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 190 op/s 2024-01-25T16:19:14.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 785119 8.01 MB/sec execute 491 sec latency 33.768 ms 2024-01-25T16:19:15.786 INFO:tasks.workunit.client.0.smithi033.stdout: 1 787124 8.01 MB/sec execute 492 sec latency 20.576 ms 2024-01-25T16:19:16.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 788498 8.02 MB/sec execute 493 sec latency 102.893 ms 2024-01-25T16:19:17.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:17 smithi033 ceph-mon[31417]: pgmap v363: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:17.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:17 smithi152 ceph-mon[36015]: pgmap v363: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:17.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:17 smithi076 ceph-mon[35642]: pgmap v363: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:17.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 790123 8.01 MB/sec execute 494 sec latency 16.435 ms 2024-01-25T16:19:18.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:18 smithi033 ceph-mon[31417]: pgmap v364: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:18.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:18 smithi152 ceph-mon[36015]: pgmap v364: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:18.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:18 smithi076 ceph-mon[35642]: pgmap v364: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:18.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 792001 8.02 MB/sec execute 495 sec latency 14.851 ms 2024-01-25T16:19:19.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 793331 8.02 MB/sec execute 496 sec latency 41.077 ms 2024-01-25T16:19:20.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 795434 8.03 MB/sec execute 497 sec latency 16.532 ms 2024-01-25T16:19:21.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:21 smithi033 ceph-mon[31417]: pgmap v365: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:21.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:21 smithi152 ceph-mon[36015]: pgmap v365: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:21.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:21 smithi076 ceph-mon[35642]: pgmap v365: 97 pgs: 97 active+clean; 116 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.4 MiB/s wr, 288 op/s 2024-01-25T16:19:21.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 796706 8.03 MB/sec execute 498 sec latency 43.942 ms 2024-01-25T16:19:22.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:22 smithi033 ceph-mon[31417]: pgmap v366: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-01-25T16:19:22.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:22 smithi152 ceph-mon[36015]: pgmap v366: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-01-25T16:19:22.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:22 smithi076 ceph-mon[35642]: pgmap v366: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.2 MiB/s wr, 329 op/s 2024-01-25T16:19:22.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 798875 8.03 MB/sec execute 499 sec latency 15.899 ms 2024-01-25T16:19:23.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 800121 8.03 MB/sec execute 500 sec latency 17.022 ms 2024-01-25T16:19:24.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:24 smithi033 ceph-mon[31417]: pgmap v367: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-01-25T16:19:24.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:24 smithi152 ceph-mon[36015]: pgmap v367: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-01-25T16:19:24.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:24 smithi076 ceph-mon[35642]: pgmap v367: 97 pgs: 97 active+clean; 122 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.9 MiB/s wr, 245 op/s 2024-01-25T16:19:24.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 802202 8.03 MB/sec execute 501 sec latency 75.086 ms 2024-01-25T16:19:25.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 803285 8.03 MB/sec execute 502 sec latency 18.527 ms 2024-01-25T16:19:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 -- 172.21.15.33:0/149377693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc104b90 msgr2=0x7f45dc104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 --2- 172.21.15.33:0/149377693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc104b90 0x7f45dc104f70 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc02bc30 comp rx=0 tx=0).stop 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 -- 172.21.15.33:0/149377693 shutdown_connections 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 --2- 172.21.15.33:0/149377693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc100800 0x7f45dc10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 --2- 172.21.15.33:0/149377693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc0ffe40 0x7f45dc1002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 --2- 172.21.15.33:0/149377693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc104b90 0x7f45dc104f70 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 -- 172.21.15.33:0/149377693 >> 172.21.15.33:0/149377693 conn(0x7f45dc076a40 msgr2=0x7f45dc076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 -- 172.21.15.33:0/149377693 shutdown_connections 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 -- 172.21.15.33:0/149377693 wait complete. 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.409+0000 7f45e2bec640 1 Processor -- start 2024-01-25T16:19:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 -- start start 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 0x7f45dc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc104b90 0x7f45dc1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc198d70 con 0x7f45dc100800 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1a3550 con 0x7f45dc0ffe40 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e2bec640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc1a3690 con 0x7f45dc104b90 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2024-01-25T16:19:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e0988640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 0x7f45dc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 -- 172.21.15.33:0/2870767139 learned_addr learned my addr 172.21.15.33:0/2870767139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e1189640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc104b90 0x7f45dc1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 -- 172.21.15.33:0/2870767139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc104b90 msgr2=0x7f45dc1a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc104b90 0x7f45dc1a3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 -- 172.21.15.33:0/2870767139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 msgr2=0x7f45dc197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 0x7f45dc197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45dbfff640 1 -- 172.21.15.33:0/2870767139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45cc009560 con 0x7f45dc100800 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.410+0000 7f45e0988640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 0x7f45dc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c950 tx=0x7f45d000ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45d9ffb640 1 -- 172.21.15.33:0/2870767139 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0007c70 con 0x7f45dc100800 2024-01-25T16:19:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc1a3940 con 0x7f45dc100800 2024-01-25T16:19:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45d9ffb640 1 -- 172.21.15.33:0/2870767139 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d0007dd0 con 0x7f45dc100800 2024-01-25T16:19:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45d9ffb640 1 -- 172.21.15.33:0/2870767139 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0018800 con 0x7f45dc100800 2024-01-25T16:19:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.411+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a3ea0 con 0x7f45dc100800 2024-01-25T16:19:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.413+0000 7f45d9ffb640 1 -- 172.21.15.33:0/2870767139 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f45d000faa0 con 0x7f45dc100800 2024-01-25T16:19:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 Processor -- start 2024-01-25T16:19:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 -- start start 2024-01-25T16:19:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45d9ffb640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 0x7f45c40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a3279f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327aab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a3270e070 con 0x559a3279f9b0 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a3270b760 con 0x559a327a8780 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e57b5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a32707050 con 0x559a3279f090 2024-01-25T16:19:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e1189640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 0x7f45c40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45d9ffb640 1 -- 172.21.15.33:0/2870767139 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f45d00cd020 con 0x7f45dc100800 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e1189640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e0988640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a3279f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49000/0 (socket says 172.21.15.33:49000) 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 -- 172.21.15.33:0/2206897809 learned_addr learned my addr 172.21.15.33:0/2206897809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 -- 172.21.15.33:0/2206897809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 msgr2=0x559a3279f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a3279f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 -- 172.21.15.33:0/2206897809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 msgr2=0x559a327aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 -- 172.21.15.33:0/2206897809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a326fee70 con 0x559a3279f9b0 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45e1189640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 0x7f45c40a39b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005fd0 tx=0x7f45d4009e80 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.414+0000 7f45dbfff640 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ffd0 tx=0x7f45d00999e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8afa65924e85af2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.415+0000 7f45c2ffd640 1 -- 172.21.15.33:0/2206897809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00774a0 con 0x559a3279f9b0 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.415+0000 7f45c2ffd640 1 -- 172.21.15.33:0/2206897809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d00b9040 con 0x559a3279f9b0 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.415+0000 7f45c2ffd640 1 -- 172.21.15.33:0/2206897809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d009df30 con 0x559a3279f9b0 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- 172.21.15.33:0/2206897809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 msgr2=0x559a327a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f45d000ffd0 tx=0x7f45d00999e0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- 172.21.15.33:0/2206897809 shutdown_connections 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327a8240 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 --2- 172.21.15.33:0/2206897809 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a3279f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- 172.21.15.33:0/2206897809 >> 172.21.15.33:0/2206897809 conn(0x559a327999b0 msgr2=0x559a32799d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- 172.21.15.33:0/2206897809 shutdown_connections 2024-01-25T16:19:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- 172.21.15.33:0/2206897809 wait complete. 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 -- start start 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.416+0000 7f45e57b5740 1 Processor -- start 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a3279f090 0x559a327a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a326c01e0 con 0x559a327a8780 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a326bfe80 con 0x559a3279f090 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a326850c0 con 0x559a3279f9b0 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 10 client.0 _ll_register_callbacks cb 0x559a326aaac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49014/0 (socket says 172.21.15.33:49014) 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e0988640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a3279f090 0x559a327a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 -- 172.21.15.33:0/196407732 learned_addr learned my addr 172.21.15.33:0/196407732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 -- 172.21.15.33:0/196407732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 msgr2=0x559a327a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 -- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a3279f090 msgr2=0x559a327a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a3279f090 0x559a327a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 -- 172.21.15.33:0/196407732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc1a3ea0 con 0x559a327a8780 2024-01-25T16:19:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45dbfff640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e1189640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009a40 tx=0x7f45d401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540d9293bdef7799 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d4016040 con 0x559a327a8780 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.417+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a32700040 con 0x559a327a8780 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f45d4003e00 con 0x559a327a8780 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d40047c0 con 0x559a327a8780 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 10 client.16350 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a327056d0 con 0x559a327a8780 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d4034020 con 0x559a327a8780 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45d4026030 con 0x559a327a8780 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 10 client.16350 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 20 client.16350 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a327b8600 0x559a327baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 4 client.16350 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a326ffd60 con 0x559a327b8600 2024-01-25T16:19:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.418+0000 7f45e0988640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a327b8600 0x559a327baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.419+0000 7f45bbfff640 10 client.16350 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.419+0000 7f45e0988640 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a327b8600 0x559a327baa20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9421f927b453e10e server_cookie=367b662ce4f54b31 in_seq=0 out_seq=0 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.419+0000 7f45bbfff640 1 -- 172.21.15.33:0/196407732 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a326ffd60 con 0x559a327b8600 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.419+0000 7f45bbfff640 10 client.16350 handle_command_reply: tid=0 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.419+0000 7f45e57b5740 1 client.16350 shutdown 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 msgr2=0x559a327a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009a40 tx=0x7f45d401d000 comp rx=0 tx=0).stop 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 shutdown_connections 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a327b8600 0x559a327baa20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a327a8780 0x559a327a61b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 --2- 172.21.15.33:0/196407732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a3279f090 0x559a327a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 >> 172.21.15.33:0/196407732 conn(0x559a327999b0 msgr2=0x559a3279b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45bbfff640 0 client.16350 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 shutdown_connections 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 1 -- 172.21.15.33:0/196407732 wait complete. 2024-01-25T16:19:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.420+0000 7f45e57b5740 20 client.16350 trim_cache size 0 max 16384 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.433+0000 7f45e57b5740 1 Processor -- start 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.433+0000 7f45e57b5740 1 -- start start 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a3279e870 con 0x559a3279f090 2024-01-25T16:19:26.442 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a327aab70 con 0x559a327a8780 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e57b5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a327a05e0 con 0x559a3279f9b0 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49030/0 (socket says 172.21.15.33:49030) 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 -- 172.21.15.33:0/1710656590 learned_addr learned my addr 172.21.15.33:0/1710656590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e1189640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 -- 172.21.15.33:0/1710656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 msgr2=0x559a327a5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 -- 172.21.15.33:0/1710656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 msgr2=0x559a327a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.443 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 -- 172.21.15.33:0/1710656590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a326a2060 con 0x559a3279f090 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45dbfff640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e1189640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45e0988640 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f0063cc893c256 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45c27fc640 1 -- 172.21.15.33:0/1710656590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc045040 con 0x559a3279f090 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.434+0000 7f45c27fc640 1 -- 172.21.15.33:0/1710656590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45cc047040 con 0x559a3279f090 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45c27fc640 1 -- 172.21.15.33:0/1710656590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004f90 con 0x559a3279f090 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 -- 172.21.15.33:0/1710656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 msgr2=0x559a327a7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc005e40 comp rx=0 tx=0).stop 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 -- 172.21.15.33:0/1710656590 shutdown_connections 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.444 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f9b0 0x559a327a5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 --2- 172.21.15.33:0/1710656590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f090 0x559a327a7e60 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 -- 172.21.15.33:0/1710656590 >> 172.21.15.33:0/1710656590 conn(0x559a327999b0 msgr2=0x559a327bf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 -- 172.21.15.33:0/1710656590 shutdown_connections 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.435+0000 7f45e57b5740 1 -- 172.21.15.33:0/1710656590 wait complete. 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 -- start start 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 Processor -- start 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a327af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a327ae4d0 con 0x559a3279f9b0 2024-01-25T16:19:26.445 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a327ae640 con 0x559a327a8780 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a327ae800 con 0x559a3279f090 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e57b5740 10 client.0 _ll_register_callbacks cb 0x7f45d000fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49032/0 (socket says 172.21.15.33:49032) 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45dbfff640 1 -- 172.21.15.33:0/731854303 learned_addr learned my addr 172.21.15.33:0/731854303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e1189640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.446 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.436+0000 7f45e0988640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a327af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 -- 172.21.15.33:0/731854303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 msgr2=0x559a327af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a327af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 -- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 msgr2=0x559a327adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 -- 172.21.15.33:0/731854303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a326fee70 con 0x559a3279f9b0 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45e1189640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45dbfff640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cad30 tx=0x7f45d0099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13fa54545f83f412 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d00994d0 con 0x559a3279f9b0 2024-01-25T16:19:26.447 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f45d009df30 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45d00b9040 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45d00a21f0 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45e57b5740 10 client.16356 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.437+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x559a327056d0 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f45d007d020 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f45d0016020 con 0x559a3279f9b0 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 10 client.16356 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 20 client.16356 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a3280fca0 0x559a328120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:26.448 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 4 client.16356 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:19:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559a32406600 con 0x559a3280fca0 2024-01-25T16:19:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e0988640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a3280fca0 0x559a328120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45e0988640 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a3280fca0 0x559a328120c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac13a5e9cb9d98f server_cookie=4b4d3f9ccbe2cc93 in_seq=0 out_seq=0 2024-01-25T16:19:26.449 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.438+0000 7f45bbfff640 10 client.16356 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45bbfff640 1 -- 172.21.15.33:0/731854303 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559a32406600 con 0x559a3280fca0 2024-01-25T16:19:26.543 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45bbfff640 10 client.16356 handle_command_reply: tid=0 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 client.16356 shutdown 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 msgr2=0x559a327ada50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f45d00cad30 tx=0x7f45d0099a50 comp rx=0 tx=0).stop 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 shutdown_connections 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x559a3280fca0 0x559a328120c0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559a327a8780 0x559a327adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45bbfff640 0 client.16356 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a3279f9b0 0x559a327ada50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 --2- 172.21.15.33:0/731854303 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x559a3279f090 0x559a327af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.542+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 >> 172.21.15.33:0/731854303 conn(0x559a327999b0 msgr2=0x559a327a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 shutdown_connections 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e57b5740 1 -- 172.21.15.33:0/731854303 wait complete. 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e57b5740 20 client.16356 trim_cache size 0 max 16384 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:19:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:19:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 msgr2=0x7f45c40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 0x7f45c40a39b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45d4005fd0 tx=0x7f45d4009e80 comp rx=0 tx=0).stop 2024-01-25T16:19:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 msgr2=0x7f45dc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f45d000c950 tx=0x7f45d000ce10 comp rx=0 tx=0).stop 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 shutdown_connections 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f45c40a14f0 0x7f45c40a39b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f45dc104b90 0x7f45dc1a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45dc100800 0x7f45dc1984d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 --2- 172.21.15.33:0/2870767139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45dc0ffe40 0x7f45dc197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.543+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 >> 172.21.15.33:0/2870767139 conn(0x7f45dc076a40 msgr2=0x7f45dc0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.544+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 shutdown_connections 2024-01-25T16:19:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:26.544+0000 7f45e2bec640 1 -- 172.21.15.33:0/2870767139 wait complete. 2024-01-25T16:19:26.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 805177 8.03 MB/sec execute 503 sec latency 48.646 ms 2024-01-25T16:19:27.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:27 smithi033 ceph-mon[31417]: pgmap v368: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-01-25T16:19:27.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:27 smithi152 ceph-mon[36015]: pgmap v368: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-01-25T16:19:27.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:27 smithi076 ceph-mon[35642]: pgmap v368: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.1 MiB/s wr, 332 op/s 2024-01-25T16:19:27.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 806515 8.03 MB/sec execute 504 sec latency 21.699 ms 2024-01-25T16:19:28.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:28 smithi033 ceph-mon[31417]: pgmap v369: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:28.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:28 smithi152 ceph-mon[36015]: pgmap v369: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:28.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:28 smithi076 ceph-mon[35642]: pgmap v369: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:28.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 808847 8.04 MB/sec execute 505 sec latency 21.207 ms 2024-01-25T16:19:29.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 810032 8.04 MB/sec execute 506 sec latency 99.416 ms 2024-01-25T16:19:30.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 812285 8.05 MB/sec execute 507 sec latency 36.610 ms 2024-01-25T16:19:31.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:31 smithi033 ceph-mon[31417]: pgmap v370: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:31.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:31 smithi152 ceph-mon[36015]: pgmap v370: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:31.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:31 smithi076 ceph-mon[35642]: pgmap v370: 97 pgs: 97 active+clean; 128 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.6 MiB/s wr, 175 op/s 2024-01-25T16:19:31.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 813638 8.05 MB/sec execute 508 sec latency 25.396 ms 2024-01-25T16:19:32.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:32 smithi033 ceph-mon[31417]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-01-25T16:19:32.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:32 smithi152 ceph-mon[36015]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-01-25T16:19:32.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:32 smithi076 ceph-mon[35642]: pgmap v371: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 325 op/s 2024-01-25T16:19:32.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 816057 8.05 MB/sec execute 509 sec latency 17.341 ms 2024-01-25T16:19:33.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 817187 8.05 MB/sec execute 510 sec latency 55.950 ms 2024-01-25T16:19:34.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:34 smithi033 ceph-mon[31417]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-01-25T16:19:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:34 smithi152 ceph-mon[36015]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-01-25T16:19:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:34 smithi076 ceph-mon[35642]: pgmap v372: 97 pgs: 97 active+clean; 118 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 235 op/s 2024-01-25T16:19:34.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 819471 8.06 MB/sec execute 511 sec latency 59.818 ms 2024-01-25T16:19:35.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 820697 8.06 MB/sec execute 512 sec latency 20.363 ms 2024-01-25T16:19:36.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:36 smithi033 ceph-mon[31417]: pgmap v373: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-01-25T16:19:36.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:36 smithi152 ceph-mon[36015]: pgmap v373: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-01-25T16:19:36.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:36 smithi076 ceph-mon[35642]: pgmap v373: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.5 MiB/s wr, 315 op/s 2024-01-25T16:19:36.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 823050 8.07 MB/sec execute 513 sec latency 18.609 ms 2024-01-25T16:19:37.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 824406 8.06 MB/sec execute 514 sec latency 17.255 ms 2024-01-25T16:19:38.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:38 smithi033 ceph-mon[31417]: pgmap v374: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:38.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:38 smithi152 ceph-mon[36015]: pgmap v374: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:38.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:38 smithi076 ceph-mon[35642]: pgmap v374: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:38.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 826624 8.07 MB/sec execute 515 sec latency 46.296 ms 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.636+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1916033136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc100f10 msgr2=0x7f9bcc1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.636+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1916033136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc100f10 0x7f9bcc1012f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009cd0 tx=0x7f9bc402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.636+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1916033136 shutdown_connections 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.636+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1916033136 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 0x7f9bcc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1916033136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc1018c0 0x7f9bcc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1916033136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc100f10 0x7f9bcc1012f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1916033136 >> 172.21.15.33:0/1916033136 conn(0x7f9bcc0fc7b0 msgr2=0x7f9bcc0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1916033136 shutdown_connections 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1916033136 wait complete. 2024-01-25T16:19:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 Processor -- start 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- start start 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 0x7f9bcc072ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 0x7f9bcc073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 0x7f9bcc1a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a59a0 con 0x7f9bcc073a70 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a5b10 con 0x7f9bcc100f10 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.637+0000 7f9bd3c32640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1a5cd0 con 0x7f9bcc105fe0 2024-01-25T16:19:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 0x7f9bcc1a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 0x7f9bcc1a53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39968/0 (socket says 172.21.15.33:39968) 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 0x7f9bcc072ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1138183482 learned_addr learned my addr 172.21.15.33:0/1138183482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd11cd640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 0x7f9bcc073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 -- 172.21.15.33:0/1138183482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 msgr2=0x7f9bcc073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 0x7f9bcc073530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 -- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 msgr2=0x7f9bcc1a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 0x7f9bcc1a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 -- 172.21.15.33:0/1138183482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc4009900 con 0x7f9bcc100f10 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 0x7f9bcc072ff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402c700 tx=0x7f9bc40325f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bc2ffd640 1 -- 172.21.15.33:0/1138183482 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4032890 con 0x7f9bcc100f10 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc1a5f50 con 0x7f9bcc100f10 2024-01-25T16:19:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bc2ffd640 1 -- 172.21.15.33:0/1138183482 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bc4003e30 con 0x7f9bcc100f10 2024-01-25T16:19:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bc2ffd640 1 -- 172.21.15.33:0/1138183482 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4004720 con 0x7f9bcc100f10 2024-01-25T16:19:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.638+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1a6380 con 0x7f9bcc100f10 2024-01-25T16:19:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bc2ffd640 1 -- 172.21.15.33:0/1138183482 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f9bc403b040 con 0x7f9bcc100f10 2024-01-25T16:19:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bd67f0740 1 Processor -- start 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bd67f0740 1 -- start start 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bc2ffd640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 0x7f9bb80a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.641+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf9780 0x555b4fdfbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fd5f070 con 0x555b4fdf0090 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fd5c760 con 0x555b4fdf9780 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fd58050 con 0x555b4fdf09b0 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bc2ffd640 1 -- 172.21.15.33:0/1138183482 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9bc40ef7c0 con 0x7f9bcc100f10 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 0x7f9bb80a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39970/0 (socket says 172.21.15.33:39970) 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd11cd640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 -- 172.21.15.33:0/3629981912 learned_addr learned my addr 172.21.15.33:0/3629981912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf9780 0x555b4fdfbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 -- 172.21.15.33:0/3629981912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 msgr2=0x555b4fdf9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 -- 172.21.15.33:0/3629981912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf9780 msgr2=0x555b4fdfbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf9780 0x555b4fdfbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 -- 172.21.15.33:0/3629981912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b4fd4fe70 con 0x555b4fdf0090 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 0x7f9bb80a39e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005940 tx=0x7f9bc800aa10 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4037040 tx=0x7f9bc40b8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc70b1636367aac server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bb7fff640 1 -- 172.21.15.33:0/3629981912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4032af0 con 0x555b4fdf0090 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bb7fff640 1 -- 172.21.15.33:0/3629981912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bc4004040 con 0x555b4fdf0090 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.642+0000 7f9bb7fff640 1 -- 172.21.15.33:0/3629981912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc40dd1e0 con 0x555b4fdf0090 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3629981912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 msgr2=0x555b4fdf0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4037040 tx=0x7f9bc40b8a90 comp rx=0 tx=0).stop 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3629981912 shutdown_connections 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf9780 0x555b4fdfbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3629981912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf0090 0x555b4fdf0470 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3629981912 >> 172.21.15.33:0/3629981912 conn(0x555b4fdea9b0 msgr2=0x555b4fdead90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3629981912 shutdown_connections 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3629981912 wait complete. 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.643+0000 7f9bd67f0740 1 -- start start 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 Processor -- start 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fd111e0 con 0x555b4fdf9780 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fd10e80 con 0x555b4fdf0090 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fcd60c0 con 0x555b4fdf09b0 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd67f0740 10 client.0 _ll_register_callbacks cb 0x555b4fcfbac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39980/0 (socket says 172.21.15.33:39980) 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1562054024 learned_addr learned my addr 172.21.15.33:0/1562054024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd11cd640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1562054024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 msgr2=0x555b4fdf6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 msgr2=0x555b4fdf6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1562054024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc1a6380 con 0x555b4fdf9780 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.644+0000 7f9bd11cd640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005cd0 tx=0x7f9bc801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f78f0c4dc2ab978 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc800ae40 con 0x555b4fdf9780 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9bc801f410 con 0x555b4fdf9780 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b4fd51040 con 0x555b4fdf9780 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc801f5d0 con 0x555b4fdf9780 2024-01-25T16:19:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 10 client.16368 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555b4fd566d0 con 0x555b4fdf9780 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bc8034020 con 0x555b4fdf9780 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9bc8030660 con 0x555b4fdf9780 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 10 client.16368 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 20 client.16368 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe09490 0x555b4fe0b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 4 client.16368 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b4fd50d60 con 0x555b4fe09490 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.645+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe09490 0x555b4fe0b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.646+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe09490 0x555b4fe0b8b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e50686ae0ad1b55b server_cookie=1e6ce93106d01dac in_seq=0 out_seq=0 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.646+0000 7f9bb57fa640 10 client.16368 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.646+0000 7f9bb57fa640 1 -- 172.21.15.33:0/1562054024 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555b4fd50d60 con 0x555b4fe09490 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.646+0000 7f9bb57fa640 10 client.16368 handle_command_reply: tid=0 2024-01-25T16:19:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.646+0000 7f9bd67f0740 1 client.16368 shutdown 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 msgr2=0x555b4fdf71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005cd0 tx=0x7f9bc801b000 comp rx=0 tx=0).stop 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 shutdown_connections 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe09490 0x555b4fe0b8b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf71c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1562054024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 >> 172.21.15.33:0/1562054024 conn(0x555b4fdea9b0 msgr2=0x555b4fdec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bb57fa640 0 client.16368 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 shutdown_connections 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1562054024 wait complete. 2024-01-25T16:19:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.647+0000 7f9bd67f0740 20 client.16368 trim_cache size 0 max 16384 2024-01-25T16:19:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.660+0000 7f9bd67f0740 1 Processor -- start 2024-01-25T16:19:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 -- start start 2024-01-25T16:19:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fe057f0 con 0x555b4fdf9780 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fde62e0 con 0x555b4fdf0090 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fe05e60 con 0x555b4fdf09b0 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39986/0 (socket says 172.21.15.33:39986) 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1720845035 learned_addr learned my addr 172.21.15.33:0/1720845035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd11cd640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1720845035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 msgr2=0x555b4fdf6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1720845035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 msgr2=0x555b4fdf8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 -- 172.21.15.33:0/1720845035 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b4fcf3060 con 0x555b4fdf9780 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd11cd640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800f830 tx=0x7f9bc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958892101c3acd8d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.661+0000 7f9bb77fe640 1 -- 172.21.15.33:0/1720845035 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc801db90 con 0x555b4fdf9780 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bb77fe640 1 -- 172.21.15.33:0/1720845035 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bc8034040 con 0x555b4fdf9780 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bb77fe640 1 -- 172.21.15.33:0/1720845035 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8004960 con 0x555b4fdf9780 2024-01-25T16:19:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1720845035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 msgr2=0x555b4fdf39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800f830 tx=0x7f9bc8004730 comp rx=0 tx=0).stop 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1720845035 shutdown_connections 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdf39a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf09b0 0x555b4fdf6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/1720845035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf0090 0x555b4fdf8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1720845035 >> 172.21.15.33:0/1720845035 conn(0x555b4fdea9b0 msgr2=0x555b4fdf5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1720845035 shutdown_connections 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- 172.21.15.33:0/1720845035 wait complete. 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.662+0000 7f9bd67f0740 1 -- start start 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 Processor -- start 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf0090 0x555b4fe003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf09b0 0x555b4fdfe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fdff250 con 0x555b4fdf9780 2024-01-25T16:19:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fdff3c0 con 0x555b4fdf09b0 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b4fdff580 con 0x555b4fdf0090 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd67f0740 10 client.0 _ll_register_callbacks cb 0x7f9bc4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39998/0 (socket says 172.21.15.33:39998) 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd11cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf09b0 0x555b4fdfe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 -- 172.21.15.33:0/3356184115 learned_addr learned my addr 172.21.15.33:0/3356184115 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf0090 0x555b4fe003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 -- 172.21.15.33:0/3356184115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf0090 msgr2=0x555b4fe003e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf0090 0x555b4fe003e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 -- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf09b0 msgr2=0x555b4fdfe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf09b0 0x555b4fdfe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.663+0000 7f9bd21cf640 1 -- 172.21.15.33:0/3356184115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b4fd566d0 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd21cf640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800fc90 tx=0x7f9bc8030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e499c2ccc327c51 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005b80 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9bc8035080 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9bc4004040 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9bc8030810 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 10 client.16374 fetch_fsmap learned FSMap version 39 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x555b4fd18a80 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bc4004040 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 39) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9bc8047070 con 0x555b4fdf9780 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 10 client.16374 fetch_fsmap finished waiting for FSMap version 39 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 20 client.16374 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe60b30 0x555b4fe62f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 4 client.16374 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555b4fa57600 con 0x555b4fe60b30 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.664+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe60b30 0x555b4fe62f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.665+0000 7f9bd19ce640 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe60b30 0x555b4fe62f20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a86636eafba8dadb server_cookie=80ba9b0eb34b8d47 in_seq=0 out_seq=0 2024-01-25T16:19:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.665+0000 7f9bb57fa640 10 client.16374 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:39.781 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bb57fa640 1 -- 172.21.15.33:0/3356184115 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555b4fa57600 con 0x555b4fe60b30 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bb57fa640 10 client.16374 handle_command_reply: tid=0 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 client.16374 shutdown 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 msgr2=0x555b4fdfed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800fc90 tx=0x7f9bc8030620 comp rx=0 tx=0).stop 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 shutdown_connections 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x555b4fe60b30 0x555b4fe62f20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bb57fa640 0 client.16374 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b4fdf9780 0x555b4fdfed10 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555b4fdf09b0 0x555b4fdfe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 --2- 172.21.15.33:0/3356184115 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x555b4fdf0090 0x555b4fe003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.782 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 >> 172.21.15.33:0/3356184115 conn(0x555b4fdea9b0 msgr2=0x555b4fdf54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 shutdown_connections 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 1 -- 172.21.15.33:0/3356184115 wait complete. 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.782+0000 7f9bd67f0740 20 client.16374 trim_cache size 0 max 16384 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:19:39.783 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 msgr2=0x7f9bb80a39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 0x7f9bb80a39e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005940 tx=0x7f9bc800aa10 comp rx=0 tx=0).stop 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 msgr2=0x7f9bcc072ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 0x7f9bcc072ff0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9bc402c700 tx=0x7f9bc40325f0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 shutdown_connections 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f9bb80a1520 0x7f9bb80a39e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bcc073a70 0x7f9bcc1a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9bcc105fe0 0x7f9bcc073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.784 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 --2- 172.21.15.33:0/1138183482 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9bcc100f10 0x7f9bcc072ff0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 >> 172.21.15.33:0/1138183482 conn(0x7f9bcc0fc7b0 msgr2=0x7f9bcc0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 shutdown_connections 2024-01-25T16:19:39.785 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:39.783+0000 7f9bd3c32640 1 -- 172.21.15.33:0/1138183482 wait complete. 2024-01-25T16:19:39.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 827672 8.07 MB/sec execute 516 sec latency 67.996 ms 2024-01-25T16:19:40.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:40 smithi033 ceph-mon[31417]: pgmap v375: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:40.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:40 smithi152 ceph-mon[36015]: pgmap v375: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:40.629 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:19:40.629 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:19:40.629 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:40 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/3552134607 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f18648b520 con 0x55f186461000 2024-01-25T16:19:40.629 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:19:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:19:40.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:40 smithi076 ceph-mon[35642]: pgmap v375: 97 pgs: 97 active+clean; 125 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 228 op/s 2024-01-25T16:19:40.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 829670 8.07 MB/sec execute 517 sec latency 32.839 ms 2024-01-25T16:19:41.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:41 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:19:41.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:41 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:19:41.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:41 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:19:41.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 831226 8.07 MB/sec execute 518 sec latency 18.621 ms 2024-01-25T16:19:42.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: osdmap e107: 12 total, 12 up, 12 in 2024-01-25T16:19:42.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/4060256335,v1:172.21.15.152:6837/4060256335] up:boot 2024-01-25T16:19:42.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:42.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:42.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:42.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:42 smithi033 ceph-mon[31417]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:42.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: osdmap e107: 12 total, 12 up, 12 in 2024-01-25T16:19:42.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/4060256335,v1:172.21.15.152:6837/4060256335] up:boot 2024-01-25T16:19:42.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:42.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:42.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:42.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:42 smithi152 ceph-mon[36015]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:42.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: osdmap e107: 12 total, 12 up, 12 in 2024-01-25T16:19:42.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/4060256335,v1:172.21.15.152:6837/4060256335] up:boot 2024-01-25T16:19:42.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:19:42.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:19:42.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:19:42.667 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:42 smithi076 ceph-mon[35642]: pgmap v377: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:42.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 833334 8.07 MB/sec execute 519 sec latency 19.716 ms 2024-01-25T16:19:43.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 834739 8.08 MB/sec execute 520 sec latency 42.935 ms 2024-01-25T16:19:44.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:44 smithi033 ceph-mon[31417]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:44.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:44 smithi152 ceph-mon[36015]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:44.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:44 smithi076 ceph-mon[35642]: pgmap v378: 97 pgs: 97 active+clean; 115 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 288 op/s 2024-01-25T16:19:44.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 836704 8.08 MB/sec execute 521 sec latency 64.772 ms 2024-01-25T16:19:45.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 838224 8.09 MB/sec execute 522 sec latency 24.179 ms 2024-01-25T16:19:46.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:46 smithi033 ceph-mon[31417]: pgmap v379: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:46.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:46 smithi152 ceph-mon[36015]: pgmap v379: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:46.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:46 smithi076 ceph-mon[35642]: pgmap v379: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:46.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 840267 8.08 MB/sec execute 523 sec latency 18.776 ms 2024-01-25T16:19:47.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 841859 8.09 MB/sec execute 524 sec latency 14.285 ms 2024-01-25T16:19:48.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:48 smithi033 ceph-mon[31417]: pgmap v380: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:48.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:48 smithi152 ceph-mon[36015]: pgmap v380: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:48.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:48 smithi076 ceph-mon[35642]: pgmap v380: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:48.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 843940 8.09 MB/sec execute 525 sec latency 21.249 ms 2024-01-25T16:19:49.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 845460 8.10 MB/sec execute 526 sec latency 16.461 ms 2024-01-25T16:19:50.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:50 smithi033 ceph-mon[31417]: pgmap v381: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:50.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:50 smithi152 ceph-mon[36015]: pgmap v381: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:50.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:50 smithi076 ceph-mon[35642]: pgmap v381: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.1 MiB/s wr, 293 op/s 2024-01-25T16:19:50.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 847415 8.10 MB/sec execute 527 sec latency 58.464 ms 2024-01-25T16:19:51.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 848965 8.10 MB/sec execute 528 sec latency 16.070 ms 2024-01-25T16:19:52.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:52 smithi033 ceph-mon[31417]: pgmap v382: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2024-01-25T16:19:52.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:52 smithi152 ceph-mon[36015]: pgmap v382: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2024-01-25T16:19:52.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:52 smithi076 ceph-mon[35642]: pgmap v382: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 190 op/s 2024-01-25T16:19:52.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 851018 8.10 MB/sec execute 529 sec latency 24.402 ms 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/1737262263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 msgr2=0x7fcc98104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/1737262263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98104f70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009ca0 tx=0x7fcc8802c0f0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/1737262263 shutdown_connections 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/1737262263 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 0x7fcc9810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/1737262263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/1737262263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98104f70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/1737262263 >> 172.21.15.33:0/1737262263 conn(0x7fcc98076a40 msgr2=0x7fcc98076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/1737262263 shutdown_connections 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/1737262263 wait complete. 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.880+0000 7fcc9eb9e640 1 Processor -- start 2024-01-25T16:19:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 -- start start 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981950d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 0x7fcc98195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98199f40 con 0x7fcc98104b90 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9819a0b0 con 0x7fcc980ffe40 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9eb9e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9819a270 con 0x7fcc98100800 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51540/0 (socket says 172.21.15.33:51540) 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2991257991 learned_addr learned my addr 172.21.15.33:0/2991257991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc97fff640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 0x7fcc98195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2991257991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 msgr2=0x7fcc98195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 0x7fcc98195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.882 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2991257991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 msgr2=0x7fcc981950d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981950d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2991257991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8000b6a0 con 0x7fcc98104b90 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.881+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e970 tx=0x7fcc9000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.882+0000 7fcc95ffb640 1 -- 172.21.15.33:0/2991257991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90009670 con 0x7fcc98104b90 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.882+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc88009900 con 0x7fcc98104b90 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.882+0000 7fcc95ffb640 1 -- 172.21.15.33:0/2991257991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc900054e0 con 0x7fcc98104b90 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.882+0000 7fcc95ffb640 1 -- 172.21.15.33:0/2991257991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004e80 con 0x7fcc98104b90 2024-01-25T16:19:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.882+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc981a1bf0 con 0x7fcc98104b90 2024-01-25T16:19:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcc95ffb640 1 -- 172.21.15.33:0/2991257991 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fcc90005020 con 0x7fcc98104b90 2024-01-25T16:19:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 Processor -- start 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 -- start start 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcc95ffb640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 0x7fcc840a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c17240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcc97fff640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 0x7fcc840a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcc95ffb640 1 -- 172.21.15.33:0/2991257991 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc90014070 con 0x7fcc98104b90 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 0x563366c19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366b7d070 con 0x563366c0e090 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366b7a760 con 0x563366c17780 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.884+0000 7fcca1834740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366b76050 con 0x563366c0e9b0 2024-01-25T16:19:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51544/0 (socket says 172.21.15.33:51544) 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 -- 172.21.15.33:0/1579713454 learned_addr learned my addr 172.21.15.33:0/1579713454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 0x563366c19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc97fff640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c17240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 -- 172.21.15.33:0/1579713454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 msgr2=0x563366c17240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c17240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 -- 172.21.15.33:0/1579713454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 msgr2=0x563366c19b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 0x563366c19b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 -- 172.21.15.33:0/1579713454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563366b6de70 con 0x563366c0e090 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 0x563366c19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc97fff640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 0x7fcc840a39b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc980770a0 tx=0x7fcc8001aa00 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802c700 tx=0x7fcc88005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2c6c023e1eef0a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc8effd640 1 -- 172.21.15.33:0/1579713454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88032440 con 0x563366c0e090 2024-01-25T16:19:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc8effd640 1 -- 172.21.15.33:0/1579713454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc88032ca0 con 0x563366c0e090 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.885+0000 7fcc8effd640 1 -- 172.21.15.33:0/1579713454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003c60 con 0x563366c0e090 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 -- 172.21.15.33:0/1579713454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 msgr2=0x563366c0e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802c700 tx=0x7fcc88005ef0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 -- 172.21.15.33:0/1579713454 shutdown_connections 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c17780 0x563366c19b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c17240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 --2- 172.21.15.33:0/1579713454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c0e470 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 -- 172.21.15.33:0/1579713454 >> 172.21.15.33:0/1579713454 conn(0x563366c089b0 msgr2=0x563366c08d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 -- 172.21.15.33:0/1579713454 shutdown_connections 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.886+0000 7fcca1834740 1 -- 172.21.15.33:0/1579713454 wait complete. 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 -- start start 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 Processor -- start 2024-01-25T16:19:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c14c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366b2f1e0 con 0x563366c17780 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366b2ee80 con 0x563366c0e090 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366af40c0 con 0x563366c0e9b0 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcca1834740 10 client.0 _ll_register_callbacks cb 0x563366b19ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51550/0 (socket says 172.21.15.33:51550) 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2257905054 learned_addr learned my addr 172.21.15.33:0/2257905054 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:52070/0 (socket says 172.21.15.33:52070) 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.887+0000 7fcc97fff640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c14c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2257905054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 msgr2=0x563366c14c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c14c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 msgr2=0x563366c146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 -- 172.21.15.33:0/2257905054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9810d3d0 con 0x563366c17780 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e940 tx=0x7fcc90004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8dfb579ea90d5b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9009e120 con 0x563366c17780 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563366b6f040 con 0x563366c17780 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc900a2360 con 0x563366c17780 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc900966e0 con 0x563366c17780 2024-01-25T16:19:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 10 client.16389 fetch_fsmap learned FSMap version 41 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563366b746d0 con 0x563366c17780 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc90078e90 con 0x563366c17780 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc9001f060 con 0x563366c17780 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 10 client.16389 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 10 client.16389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 20 client.16389 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.888+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c27570 0x563366c29990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.889+0000 7fcca1834740 4 client.16389 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.889+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563366b6ed60 con 0x563366c27570 2024-01-25T16:19:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.889+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c27570 0x563366c29990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.889+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c27570 0x563366c29990 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1afbe70eb43f52e server_cookie=3980372d3b4cf9ef in_seq=0 out_seq=0 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.889+0000 7fcc77fff640 10 client.16389 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcc77fff640 1 -- 172.21.15.33:0/2257905054 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563366b6ed60 con 0x563366c27570 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcc77fff640 10 client.16389 handle_command_reply: tid=0 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 client.16389 shutdown 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 msgr2=0x563366c15160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e940 tx=0x7fcc90004780 comp rx=0 tx=0).stop 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 shutdown_connections 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c27570 0x563366c29990 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c17780 0x563366c15160 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c0e9b0 0x563366c14c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 --2- 172.21.15.33:0/2257905054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e090 0x563366c146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcc77fff640 0 client.16389 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 >> 172.21.15.33:0/2257905054 conn(0x563366c089b0 msgr2=0x563366c0a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 shutdown_connections 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 1 -- 172.21.15.33:0/2257905054 wait complete. 2024-01-25T16:19:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.890+0000 7fcca1834740 20 client.16389 trim_cache size 0 max 16384 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 Processor -- start 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 -- start start 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c16e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c19b70 con 0x563366c0e090 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c04270 con 0x563366c0e9b0 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcca1834740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c12c70 con 0x563366c17780 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c16e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c16e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51564/0 (socket says 172.21.15.33:51564) 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcc9c93a640 1 -- 172.21.15.33:0/3220824851 learned_addr learned my addr 172.21.15.33:0/3220824851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.906+0000 7fcc97fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c14f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 -- 172.21.15.33:0/3220824851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 msgr2=0x563366c14f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c14f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 -- 172.21.15.33:0/3220824851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 msgr2=0x563366c16e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c16e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 -- 172.21.15.33:0/3220824851 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563366b11060 con 0x563366c17780 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c119a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc900caf20 tx=0x7fcc90009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71142d9e035fc0d server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc8e7fc640 1 -- 172.21.15.33:0/3220824851 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90099b10 con 0x563366c17780 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc8e7fc640 1 -- 172.21.15.33:0/3220824851 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc90078e90 con 0x563366c17780 2024-01-25T16:19:52.907 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.907+0000 7fcc8e7fc640 1 -- 172.21.15.33:0/3220824851 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9009e120 con 0x563366c17780 2024-01-25T16:19:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- 172.21.15.33:0/3220824851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 msgr2=0x563366c119a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c119a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc900caf20 tx=0x7fcc90009f50 comp rx=0 tx=0).stop 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- 172.21.15.33:0/3220824851 shutdown_connections 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c119a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c14f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 --2- 172.21.15.33:0/3220824851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c16e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- 172.21.15.33:0/3220824851 >> 172.21.15.33:0/3220824851 conn(0x563366c089b0 msgr2=0x563366c12ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- 172.21.15.33:0/3220824851 shutdown_connections 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- 172.21.15.33:0/3220824851 wait complete. 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 -- start start 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.908+0000 7fcca1834740 1 Processor -- start 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c1d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c1d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c7daa0 con 0x563366c0e090 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c7dbe0 con 0x563366c0e9b0 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x563366c7dd20 con 0x563366c17780 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcca1834740 10 client.0 _ll_register_callbacks cb 0x7fcc80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c1d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9c93a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51576/0 (socket says 172.21.15.33:51576) 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9c93a640 1 -- 172.21.15.33:0/1492824074 learned_addr learned my addr 172.21.15.33:0/1492824074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc97fff640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c1d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 -- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 msgr2=0x563366c1d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c1d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 -- 172.21.15.33:0/1492824074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 msgr2=0x563366c1ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 -- 172.21.15.33:0/1492824074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563366b746d0 con 0x563366c17780 2024-01-25T16:19:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc9d13b640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c1d8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc90099f60 tx=0x7fcc90096420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75a67899b1d76b71 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.909+0000 7fcc77fff640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90099b10 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcc900994c0 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc88032ca0 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc9009e120 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 10 client.25918 fetch_fsmap learned FSMap version 41 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x563366b36a80 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcc88032ca0 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcc9001f040 con 0x563366c17780 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 10 client.25918 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 20 client.25918 populate_metadata read hostname 'smithi033' 2024-01-25T16:19:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c7f000 0x563366c813f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:19:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 4 client.25918 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:19:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563366875600 con 0x563366c7f000 2024-01-25T16:19:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.910+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c7f000 0x563366c813f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:19:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.911+0000 7fcc77fff640 10 client.25918 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:52.911+0000 7fcc9c93a640 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c7f000 0x563366c813f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4f51c563c77e6c server_cookie=631735ef8550d39e in_seq=0 out_seq=0 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcc77fff640 1 -- 172.21.15.33:0/1492824074 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563366875600 con 0x563366c7f000 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcc77fff640 10 client.25918 handle_command_reply: tid=0 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 client.25918 shutdown 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 msgr2=0x563366c1d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c1d8b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcc90099f60 tx=0x7fcc90096420 comp rx=0 tx=0).stop 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 shutdown_connections 2024-01-25T16:19:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x563366c7f000 0x563366c813f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x563366c17780 0x563366c1d8b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.033+0000 7fcc77fff640 0 client.25918 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563366c0e9b0 0x563366c1d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 1 --2- 172.21.15.33:0/1492824074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563366c0e090 0x563366c1ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 >> 172.21.15.33:0/1492824074 conn(0x563366c089b0 msgr2=0x563366c071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 shutdown_connections 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 1 -- 172.21.15.33:0/1492824074 wait complete. 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.034+0000 7fcca1834740 20 client.25918 trim_cache size 0 max 16384 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:19:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:19:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:19:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 msgr2=0x7fcc840a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 0x7fcc840a39b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcc980770a0 tx=0x7fcc8001aa00 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 msgr2=0x7fcc98199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000e970 tx=0x7fcc9000ee30 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 shutdown_connections 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fcc840a14f0 0x7fcc840a39b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc98104b90 0x7fcc98199940 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc98100800 0x7fcc98195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 --2- 172.21.15.33:0/2991257991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc980ffe40 0x7fcc981950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 >> 172.21.15.33:0/2991257991 conn(0x7fcc98076a40 msgr2=0x7fcc980fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 shutdown_connections 2024-01-25T16:19:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:19:53.035+0000 7fcc9eb9e640 1 -- 172.21.15.33:0/2991257991 wait complete. 2024-01-25T16:19:53.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 852495 8.11 MB/sec execute 530 sec latency 43.064 ms 2024-01-25T16:19:54.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:54 smithi033 ceph-mon[31417]: pgmap v383: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-01-25T16:19:54.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:54 smithi152 ceph-mon[36015]: pgmap v383: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-01-25T16:19:54.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:54 smithi076 ceph-mon[35642]: pgmap v383: 97 pgs: 97 active+clean; 127 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-01-25T16:19:54.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 854541 8.11 MB/sec execute 531 sec latency 22.946 ms 2024-01-25T16:19:55.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 855909 8.11 MB/sec execute 532 sec latency 50.233 ms 2024-01-25T16:19:56.563 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:56 smithi033 ceph-mon[31417]: pgmap v384: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:19:56.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:56 smithi152 ceph-mon[36015]: pgmap v384: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:19:56.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:56 smithi076 ceph-mon[35642]: pgmap v384: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 326 op/s 2024-01-25T16:19:56.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 857479 8.11 MB/sec execute 533 sec latency 17.721 ms 2024-01-25T16:19:57.787 INFO:tasks.workunit.client.0.smithi033.stdout: 1 859409 8.12 MB/sec execute 534 sec latency 25.813 ms 2024-01-25T16:19:58.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:19:58 smithi033 ceph-mon[31417]: pgmap v385: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:19:58.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:19:58 smithi152 ceph-mon[36015]: pgmap v385: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:19:58.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:19:58 smithi076 ceph-mon[35642]: pgmap v385: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:19:58.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 860898 8.11 MB/sec execute 535 sec latency 17.097 ms 2024-01-25T16:19:59.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 862968 8.12 MB/sec execute 536 sec latency 18.352 ms 2024-01-25T16:20:00.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:00 smithi033 ceph-mon[31417]: overall HEALTH_OK 2024-01-25T16:20:00.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:00 smithi152 ceph-mon[36015]: overall HEALTH_OK 2024-01-25T16:20:00.414 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:00 smithi076 ceph-mon[35642]: overall HEALTH_OK 2024-01-25T16:20:00.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 863752 8.11 MB/sec execute 537 sec latency 241.485 ms 2024-01-25T16:20:01.490 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:01 smithi152 ceph-mon[36015]: pgmap v386: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:20:01.490 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:01 smithi076 ceph-mon[35642]: pgmap v386: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:20:01.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:01 smithi033 ceph-mon[31417]: pgmap v386: 97 pgs: 97 active+clean; 121 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 242 op/s 2024-01-25T16:20:01.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 865213 8.11 MB/sec execute 538 sec latency 26.187 ms 2024-01-25T16:20:02.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 866378 8.11 MB/sec execute 539 sec latency 36.430 ms 2024-01-25T16:20:03.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:03 smithi152 ceph-mon[36015]: osdmap e108: 12 total, 12 up, 12 in 2024-01-25T16:20:03.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:03 smithi152 ceph-mon[36015]: pgmap v388: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:03.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:03 smithi076 ceph-mon[35642]: osdmap e108: 12 total, 12 up, 12 in 2024-01-25T16:20:03.415 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:03 smithi076 ceph-mon[35642]: pgmap v388: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:03.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:03 smithi033 ceph-mon[31417]: osdmap e108: 12 total, 12 up, 12 in 2024-01-25T16:20:03.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:03 smithi033 ceph-mon[31417]: pgmap v388: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:03.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 866886 8.10 MB/sec execute 540 sec latency 33.493 ms 2024-01-25T16:20:04.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:04 smithi033 ceph-mon[31417]: pgmap v389: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:04.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:04 smithi152 ceph-mon[36015]: pgmap v389: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:04.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:04 smithi076 ceph-mon[35642]: pgmap v389: 97 pgs: 97 active+clean; 127 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 321 op/s 2024-01-25T16:20:04.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 869207 8.11 MB/sec execute 541 sec latency 19.148 ms 2024-01-25T16:20:05.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 870320 8.11 MB/sec execute 542 sec latency 106.197 ms 2024-01-25T16:20:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 -- 172.21.15.33:0/455532388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068370 msgr2=0x7ffbf40687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/455532388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068370 0x7ffbf40687d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009900 tx=0x7ffbe802bc30 comp rx=0 tx=0).stop 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 -- 172.21.15.33:0/455532388 shutdown_connections 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/455532388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068d10 0x7ffbf410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/455532388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068370 0x7ffbf40687d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/455532388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf41071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 -- 172.21.15.33:0/455532388 >> 172.21.15.33:0/455532388 conn(0x7ffbf4076a40 msgr2=0x7ffbf4076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 -- 172.21.15.33:0/455532388 shutdown_connections 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.124+0000 7ffbfa91d640 1 -- 172.21.15.33:0/455532388 wait complete. 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 Processor -- start 2024-01-25T16:20:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 -- start start 2024-01-25T16:20:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 0x7ffbf4197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 0x7ffbf4197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf419bfe0 con 0x7ffbf4106e00 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf419c150 con 0x7ffbf4068d10 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbfa91d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbf419c310 con 0x7ffbf4068370 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbf8eba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 0x7ffbf4197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbf8eba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43712/0 (socket says 172.21.15.33:43712) 2024-01-25T16:20:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbf8eba640 1 -- 172.21.15.33:0/3672035894 learned_addr learned my addr 172.21.15.33:0/3672035894 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.125+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 0x7ffbf4197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 -- 172.21.15.33:0/3672035894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 msgr2=0x7ffbf4197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 0x7ffbf4197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 -- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 msgr2=0x7ffbf419b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 -- 172.21.15.33:0/3672035894 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbec009840 con 0x7ffbf4068370 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 0x7ffbf4197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 0x7ffbf4197120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00ec80 tx=0x7ffbec00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf17fa640 1 -- 172.21.15.33:0/3672035894 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec003b70 con 0x7ffbf4068370 2024-01-25T16:20:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe8009560 con 0x7ffbf4068370 2024-01-25T16:20:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf41a3c90 con 0x7ffbf4068370 2024-01-25T16:20:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf17fa640 1 -- 172.21.15.33:0/3672035894 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbec003cd0 con 0x7ffbf4068370 2024-01-25T16:20:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.126+0000 7ffbf17fa640 1 -- 172.21.15.33:0/3672035894 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec010850 con 0x7ffbf4068370 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf17fa640 1 -- 172.21.15.33:0/3672035894 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7ffbec003e40 con 0x7ffbf4068370 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 Processor -- start 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 -- start start 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee58470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee63b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2edc7070 con 0x561e2ee589b0 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2edc4760 con 0x561e2ee61780 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2edc0050 con 0x561e2ee58090 2024-01-25T16:20:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43714/0 (socket says 172.21.15.33:43714) 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2672602796 learned_addr learned my addr 172.21.15.33:0/2672602796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee63b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee58470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf17fa640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 0x7ffbdc0a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 0x7ffbdc0a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2672602796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 msgr2=0x561e2ee58470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee58470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2672602796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 msgr2=0x561e2ee63b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee63b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.129+0000 7ffbf17fa640 1 -- 172.21.15.33:0/3672035894 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ffbec014070 con 0x7ffbf4068370 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2672602796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2edb7e70 con 0x561e2ee589b0 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffbec09a4f0 tx=0x7ffbec00ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422f1bbcbcd0c479 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 0x7ffbdc0a3a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009fd0 tx=0x7ffbe4004730 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbda7fc640 1 -- 172.21.15.33:0/2672602796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec099ec0 con 0x561e2ee589b0 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbda7fc640 1 -- 172.21.15.33:0/2672602796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbec09ac60 con 0x561e2ee589b0 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbda7fc640 1 -- 172.21.15.33:0/2672602796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec096e10 con 0x561e2ee589b0 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2672602796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 msgr2=0x561e2ee61240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.130+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ffbec09a4f0 tx=0x7ffbec00ef80 comp rx=0 tx=0).stop 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2672602796 shutdown_connections 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee63b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee61240 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2672602796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee58470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2672602796 >> 172.21.15.33:0/2672602796 conn(0x561e2ee529b0 msgr2=0x561e2ee52d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2672602796 shutdown_connections 2024-01-25T16:20:06.139 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2672602796 wait complete. 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 -- start start 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.131+0000 7ffbfd5b8740 1 Processor -- start 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee589b0 0x561e2ee5ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2ed791e0 con 0x561e2ee61780 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2ed78e80 con 0x561e2ee589b0 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2ed3e0c0 con 0x561e2ee58090 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 10 client.0 _ll_register_callbacks cb 0x561e2ed63ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf8eba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:56326/0 (socket says 172.21.15.33:56326) 2024-01-25T16:20:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee589b0 0x561e2ee5ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf8eba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43716/0 (socket says 172.21.15.33:43716) 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2608920852 learned_addr learned my addr 172.21.15.33:0/2608920852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee589b0 msgr2=0x561e2ee5ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee589b0 0x561e2ee5ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2608920852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 msgr2=0x561e2ee5f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 -- 172.21.15.33:0/2608920852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbf41a3c90 con 0x561e2ee58090 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0caf60 tx=0x7ffbec099ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d62bf3806665493 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec0cb340 con 0x561e2ee58090 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ffbec09e6b0 con 0x561e2ee58090 2024-01-25T16:20:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.132+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e2edb9040 con 0x561e2ee58090 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbec0a28f0 con 0x561e2ee58090 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 10 client.25930 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561e2edbe6d0 con 0x561e2ee58090 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbec0b6020 con 0x561e2ee58090 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbec01f070 con 0x561e2ee58090 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 10 client.25930 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 10 client.25930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 20 client.25930 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2ee711a0 0x561e2ee73590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 4 client.25930 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e2edb8d60 con 0x561e2ee711a0 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2ee711a0 0x561e2ee73590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2ee711a0 0x561e2ee73590 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf1447393c8fb20 server_cookie=71c8eb9334a20e73 in_seq=0 out_seq=0 2024-01-25T16:20:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.133+0000 7ffbd37fe640 10 client.25930 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbd37fe640 1 -- 172.21.15.33:0/2608920852 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e2edb8d60 con 0x561e2ee711a0 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbd37fe640 10 client.25930 handle_command_reply: tid=0 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 client.25930 shutdown 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 msgr2=0x561e2ee5e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffbec0caf60 tx=0x7ffbec099ec0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 shutdown_connections 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2ee711a0 0x561e2ee73590 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee61780 0x561e2ee5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee589b0 0x561e2ee5ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/2608920852 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee5e6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbd37fe640 0 client.25930 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 >> 172.21.15.33:0/2608920852 conn(0x561e2ee529b0 msgr2=0x561e2ee54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 shutdown_connections 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/2608920852 wait complete. 2024-01-25T16:20:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.134+0000 7ffbfd5b8740 20 client.25930 trim_cache size 0 max 16384 2024-01-25T16:20:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 Processor -- start 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 -- start start 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee60d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee5d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2edb8d60 con 0x561e2ee589b0 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2ee537b0 con 0x561e2ee61780 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2ee63bc0 con 0x561e2ee58090 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.148+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43722/0 (socket says 172.21.15.33:43722) 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 -- 172.21.15.33:0/3008082089 learned_addr learned my addr 172.21.15.33:0/3008082089 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee60d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee5d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 -- 172.21.15.33:0/3008082089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 msgr2=0x561e2ee60d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee60d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 -- 172.21.15.33:0/3008082089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 msgr2=0x561e2ee5d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee5d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 -- 172.21.15.33:0/3008082089 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e2ed5b060 con 0x561e2ee589b0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009900 tx=0x7ffbe802c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4930bfb91a36f8b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbd9ffb640 1 -- 172.21.15.33:0/3008082089 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe8045040 con 0x561e2ee589b0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbd9ffb640 1 -- 172.21.15.33:0/3008082089 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbe8047040 con 0x561e2ee589b0 2024-01-25T16:20:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.149+0000 7ffbd9ffb640 1 -- 172.21.15.33:0/3008082089 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbe804c740 con 0x561e2ee589b0 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/3008082089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 msgr2=0x561e2ee5ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ffbe8009900 tx=0x7ffbe802c6c0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/3008082089 shutdown_connections 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee5d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee589b0 0x561e2ee5ef50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/3008082089 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee58090 0x561e2ee60d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/3008082089 >> 172.21.15.33:0/3008082089 conn(0x561e2ee529b0 msgr2=0x561e2ee5cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/3008082089 shutdown_connections 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/3008082089 wait complete. 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 -- start start 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.150+0000 7ffbfd5b8740 1 Processor -- start 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee589b0 0x561e2ee674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee679e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2eec7620 con 0x561e2ee58090 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2eec7760 con 0x561e2ee61780 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e2eec78a0 con 0x561e2ee589b0 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 10 client.0 _ll_register_callbacks cb 0x7ffbec0966d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43724/0 (socket says 172.21.15.33:43724) 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 -- 172.21.15.33:0/1629330979 learned_addr learned my addr 172.21.15.33:0/1629330979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf8eba640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee679e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee589b0 0x561e2ee674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 -- 172.21.15.33:0/1629330979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee589b0 msgr2=0x561e2ee674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee589b0 0x561e2ee674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 -- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 msgr2=0x561e2ee679e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee679e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 -- 172.21.15.33:0/1629330979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe8030070 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbf3fff640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffbec09a3b0 tx=0x7ffbec096660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0f843eb5288395b server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbec083d40 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ffbec0b7080 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.151+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffbec09ac60 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffbec083ea0 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 10 client.16413 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561e2ed80a80 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbec09ac60 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffbec0994c0 con 0x561e2ee58090 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 10 client.16413 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 20 client.16413 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2eec8b40 0x561e2eecaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 4 client.16413 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e2eabf600 con 0x561e2eec8b40 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.152+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2eec8b40 0x561e2eecaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.153+0000 7ffbd37fe640 10 client.16413 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.153+0000 7ffbf37fe640 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2eec8b40 0x561e2eecaf30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3fe5e2e11bc7f4c server_cookie=e328a51a7640044e in_seq=0 out_seq=0 2024-01-25T16:20:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbd37fe640 1 -- 172.21.15.33:0/1629330979 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e2eabf600 con 0x561e2eec8b40 2024-01-25T16:20:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbd37fe640 10 client.16413 handle_command_reply: tid=0 2024-01-25T16:20:06.286 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 client.16413 shutdown 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 msgr2=0x561e2ee66f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffbec09a3b0 tx=0x7ffbec096660 comp rx=0 tx=0).stop 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 shutdown_connections 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561e2eec8b40 0x561e2eecaf30 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e2ee61780 0x561e2ee679e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561e2ee589b0 0x561e2ee674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbd37fe640 0 client.16413 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.287+0000 7ffbfd5b8740 1 --2- 172.21.15.33:0/1629330979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e2ee58090 0x561e2ee66f60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 >> 172.21.15.33:0/1629330979 conn(0x561e2ee529b0 msgr2=0x561e2ee51a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.287 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 shutdown_connections 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfd5b8740 1 -- 172.21.15.33:0/1629330979 wait complete. 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfd5b8740 20 client.16413 trim_cache size 0 max 16384 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:20:06.288 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:20:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 msgr2=0x7ffbdc0a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 0x7ffbdc0a3a30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbe4009fd0 tx=0x7ffbe4004730 comp rx=0 tx=0).stop 2024-01-25T16:20:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.288+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 msgr2=0x7ffbf4197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:06.289 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 0x7ffbf4197120 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffbec00ec80 tx=0x7ffbec00c6a0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 shutdown_connections 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7ffbdc0a1570 0x7ffbdc0a3a30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffbf4106e00 0x7ffbf419b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffbf4068d10 0x7ffbf4197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 --2- 172.21.15.33:0/3672035894 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffbf4068370 0x7ffbf4197120 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 >> 172.21.15.33:0/3672035894 conn(0x7ffbf4076a40 msgr2=0x7ffbf40fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 shutdown_connections 2024-01-25T16:20:06.290 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:06.289+0000 7ffbfa91d640 1 -- 172.21.15.33:0/3672035894 wait complete. 2024-01-25T16:20:06.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:06 smithi033 ceph-mon[31417]: pgmap v390: 97 pgs: 97 active+clean; 130 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:20:06.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:06 smithi152 ceph-mon[36015]: pgmap v390: 97 pgs: 97 active+clean; 130 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:20:06.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:06 smithi076 ceph-mon[35642]: pgmap v390: 97 pgs: 97 active+clean; 130 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 230 op/s 2024-01-25T16:20:06.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 872268 8.10 MB/sec execute 543 sec latency 77.932 ms 2024-01-25T16:20:07.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 873812 8.11 MB/sec execute 544 sec latency 21.371 ms 2024-01-25T16:20:08.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:08 smithi033 ceph-mon[31417]: pgmap v391: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:08.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:08 smithi152 ceph-mon[36015]: pgmap v391: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:08.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:08 smithi076 ceph-mon[35642]: pgmap v391: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:08.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 875827 8.11 MB/sec execute 545 sec latency 16.249 ms 2024-01-25T16:20:09.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 877297 8.12 MB/sec execute 546 sec latency 25.770 ms 2024-01-25T16:20:10.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:10 smithi033 ceph-mon[31417]: pgmap v392: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:10.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:10 smithi033 ceph-mon[31417]: osdmap e109: 12 total, 12 up, 12 in 2024-01-25T16:20:10.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:10 smithi152 ceph-mon[36015]: pgmap v392: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:10.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:10 smithi152 ceph-mon[36015]: osdmap e109: 12 total, 12 up, 12 in 2024-01-25T16:20:10.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:10 smithi076 ceph-mon[35642]: pgmap v392: 97 pgs: 97 active+clean; 132 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.2 MiB/s wr, 249 op/s 2024-01-25T16:20:10.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:10 smithi076 ceph-mon[35642]: osdmap e109: 12 total, 12 up, 12 in 2024-01-25T16:20:10.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 878702 8.11 MB/sec execute 547 sec latency 60.918 ms 2024-01-25T16:20:11.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 880592 8.12 MB/sec execute 548 sec latency 23.316 ms 2024-01-25T16:20:12.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:12 smithi152 ceph-mon[36015]: pgmap v394: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:12.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:12 smithi076 ceph-mon[35642]: pgmap v394: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:12.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 881995 8.12 MB/sec execute 549 sec latency 17.352 ms 2024-01-25T16:20:12.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:12 smithi033 ceph-mon[31417]: pgmap v394: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:13.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 884108 8.12 MB/sec execute 550 sec latency 14.789 ms 2024-01-25T16:20:14.414 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:14 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:20:14.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:14 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:20:14.561 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:14 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:20:14.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 885275 8.12 MB/sec execute 551 sec latency 57.631 ms 2024-01-25T16:20:15.489 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:15 smithi076 ceph-mon[35642]: pgmap v395: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:15.489 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:15 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.489 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:15 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:15 smithi033 ceph-mon[31417]: pgmap v395: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:15.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:15 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:15 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:15 smithi152 ceph-mon[36015]: pgmap v395: 97 pgs: 97 active+clean; 121 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 277 op/s 2024-01-25T16:20:15.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:15 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:15 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:15.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 887357 8.12 MB/sec execute 552 sec latency 93.324 ms 2024-01-25T16:20:16.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 888231 8.12 MB/sec execute 553 sec latency 20.146 ms 2024-01-25T16:20:17.378 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: pgmap v396: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-01-25T16:20:17.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:20:17.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:20:17.379 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: pgmap v396: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:20:17.486 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: pgmap v396: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 270 op/s 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:20:17.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:20:17.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 889395 8.11 MB/sec execute 554 sec latency 18.554 ms 2024-01-25T16:20:18.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:18 smithi033 ceph-mon[31417]: pgmap v397: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:18.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:18 smithi152 ceph-mon[36015]: pgmap v397: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:18.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 891255 8.12 MB/sec execute 555 sec latency 16.540 ms 2024-01-25T16:20:18.790 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:18 smithi076 ceph-mon[35642]: pgmap v397: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 -- 172.21.15.33:0/2268261693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21940ff880 msgr2=0x7f21940ffc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 --2- 172.21.15.33:0/2268261693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21940ff880 0x7f21940ffc60 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2184009900 tx=0x7f218402bc30 comp rx=0 tx=0).stop 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 -- 172.21.15.33:0/2268261693 shutdown_connections 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 --2- 172.21.15.33:0/2268261693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f219410d430 0x7f219410f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 --2- 172.21.15.33:0/2268261693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f219410cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 --2- 172.21.15.33:0/2268261693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21940ff880 0x7f21940ffc60 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 -- 172.21.15.33:0/2268261693 >> 172.21.15.33:0/2268261693 conn(0x7f21940fb500 msgr2=0x7f21940fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 -- 172.21.15.33:0/2268261693 shutdown_connections 2024-01-25T16:20:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.382+0000 7f219959e640 1 -- 172.21.15.33:0/2268261693 wait complete. 2024-01-25T16:20:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 Processor -- start 2024-01-25T16:20:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 -- start start 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f21941997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219419e110 con 0x7f219410d430 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219419e280 con 0x7f2194100230 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.383+0000 7f219959e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219419e440 con 0x7f21940ff880 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2193fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2193fff640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:57374/0 (socket says 172.21.15.33:57374) 2024-01-25T16:20:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56538/0 (socket says 172.21.15.33:56538) 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2193fff640 1 -- 172.21.15.33:0/107928238 learned_addr learned my addr 172.21.15.33:0/107928238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f21937fe640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f21941997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 -- 172.21.15.33:0/107928238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 msgr2=0x7f21941992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 -- 172.21.15.33:0/107928238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 msgr2=0x7f21941997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f21941997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 -- 172.21.15.33:0/107928238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2184009560 con 0x7f219410d430 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2193fff640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f21937fe640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f21941997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f2198d9d640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f218c005fd0 tx=0x7f218c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f21917fa640 1 -- 172.21.15.33:0/107928238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c00cdf0 con 0x7f219410d430 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219419e720 con 0x7f219410d430 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f21917fa640 1 -- 172.21.15.33:0/107928238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218c005020 con 0x7f219410d430 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f21917fa640 1 -- 172.21.15.33:0/107928238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c010670 con 0x7f219410d430 2024-01-25T16:20:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.384+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21941a5fa0 con 0x7f219410d430 2024-01-25T16:20:19.392 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21917fa640 1 -- 172.21.15.33:0/107928238 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f218c005450 con 0x7f219410d430 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 Processor -- start 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 -- start start 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 0x56152cad6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cad6950 0x56152cad8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ca3c070 con 0x56152cacd260 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ca39760 con 0x56152cad6950 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f219d3c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ca35050 con 0x56152cacdb80 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21917fa640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 0x7f217c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56548/0 (socket says 172.21.15.33:56548) 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21937fe640 1 -- 172.21.15.33:0/1635965545 learned_addr learned my addr 172.21.15.33:0/1635965545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:19.393 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f2193fff640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 0x56152cad6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f2198d9d640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 0x7f217c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.387+0000 7f21937fe640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cad6950 0x56152cad8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21917fa640 1 -- 172.21.15.33:0/107928238 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f218c014070 con 0x7f219410d430 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 -- 172.21.15.33:0/1635965545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 msgr2=0x56152cad6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 0x56152cad6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 -- 172.21.15.33:0/1635965545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cad6950 msgr2=0x56152cad8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cad6950 0x56152cad8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 -- 172.21.15.33:0/1635965545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152ca2ce70 con 0x56152cacd260 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f2193fff640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 0x56152cad6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f2198d9d640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 0x7f217c0a3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f218c0a2250 tx=0x7f218c06c450 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f21937fe640 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2188009e80 tx=0x7f21880096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0531729fd1997f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f217a7fc640 1 -- 172.21.15.33:0/1635965545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188018070 con 0x56152cacd260 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f217a7fc640 1 -- 172.21.15.33:0/1635965545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f218800f990 con 0x56152cacd260 2024-01-25T16:20:19.394 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.388+0000 7f217a7fc640 1 -- 172.21.15.33:0/1635965545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21880117b0 con 0x56152cacd260 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- 172.21.15.33:0/1635965545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 msgr2=0x56152cacd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2188009e80 tx=0x7f21880096e0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- 172.21.15.33:0/1635965545 shutdown_connections 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cad6950 0x56152cad8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cacdb80 0x56152cad6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1635965545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cacd640 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- 172.21.15.33:0/1635965545 >> 172.21.15.33:0/1635965545 conn(0x56152cac7b80 msgr2=0x56152cac7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- 172.21.15.33:0/1635965545 shutdown_connections 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- 172.21.15.33:0/1635965545 wait complete. 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 -- start start 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.389+0000 7f219d3c0740 1 Processor -- start 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.395 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152c9ee1e0 con 0x56152cacd260 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152c9ede80 con 0x56152cacdb80 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152c9b30c0 con 0x56152cad6950 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 10 client.0 _ll_register_callbacks cb 0x56152c9d8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56558/0 (socket says 172.21.15.33:56558) 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 -- 172.21.15.33:0/1252995379 learned_addr learned my addr 172.21.15.33:0/1252995379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f2193fff640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 -- 172.21.15.33:0/1252995379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 msgr2=0x56152cad4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 -- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 msgr2=0x56152cad3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.396 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 -- 172.21.15.33:0/1252995379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2188013400 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21937fe640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f218800f910 tx=0x7f2188005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3ea013800cd49 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21737fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188018070 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f218800fe30 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.390+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56152ca2e040 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2188004460 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 10 client.16425 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56152ca336d0 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56152ca2e040 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2188004bb0 con 0x56152cacd260 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 10 client.16425 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:19.397 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 20 client.16425 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cae6580 0x56152cae8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 4 client.16425 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56152ca2dd60 con 0x56152cae6580 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.391+0000 7f2193fff640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cae6580 0x56152cae8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f2193fff640 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cae6580 0x56152cae8970 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fe243c540177303 server_cookie=7cd38d30566869ce in_seq=0 out_seq=0 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f21737fe640 10 client.16425 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f21737fe640 1 -- 172.21.15.33:0/1252995379 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56152ca2dd60 con 0x56152cae6580 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f21737fe640 10 client.16425 handle_command_reply: tid=0 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 client.16425 shutdown 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 msgr2=0x56152cad38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f218800f910 tx=0x7f2188005fd0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 shutdown_connections 2024-01-25T16:20:19.398 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cae6580 0x56152cae8970 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f21737fe640 0 client.16425 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 --2- 172.21.15.33:0/1252995379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad38e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.392+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 >> 172.21.15.33:0/1252995379 conn(0x56152cac7b80 msgr2=0x56152cac98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.393+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 shutdown_connections 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.393+0000 7f219d3c0740 1 -- 172.21.15.33:0/1252995379 wait complete. 2024-01-25T16:20:19.399 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.393+0000 7f219d3c0740 20 client.16425 trim_cache size 0 max 16384 2024-01-25T16:20:19.405 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 Processor -- start 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 -- start start 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cac7420 con 0x56152cacd260 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cac34b0 con 0x56152cacdb80 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f219d3c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cac79f0 con 0x56152cad6950 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f2193fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56574/0 (socket says 172.21.15.33:56574) 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.406+0000 7f21937fe640 1 -- 172.21.15.33:0/3702482868 learned_addr learned my addr 172.21.15.33:0/3702482868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.406 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 -- 172.21.15.33:0/3702482868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 msgr2=0x56152cad10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 -- 172.21.15.33:0/3702482868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 msgr2=0x56152cad0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 -- 172.21.15.33:0/3702482868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152c9d0060 con 0x56152cacd260 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f21937fe640 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2188009690 tx=0x7f2188009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e48b07e73552760 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f2179ffb640 1 -- 172.21.15.33:0/3702482868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188018070 con 0x56152cacd260 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f2179ffb640 1 -- 172.21.15.33:0/3702482868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f21880058b0 con 0x56152cacd260 2024-01-25T16:20:19.407 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.407+0000 7f2179ffb640 1 -- 172.21.15.33:0/3702482868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21880039d0 con 0x56152cacd260 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- 172.21.15.33:0/3702482868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 msgr2=0x56152cad6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2188009690 tx=0x7f2188009960 comp rx=0 tx=0).stop 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- 172.21.15.33:0/3702482868 shutdown_connections 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cad10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cad0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 --2- 172.21.15.33:0/3702482868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cad6010 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.411 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- 172.21.15.33:0/3702482868 >> 172.21.15.33:0/3702482868 conn(0x56152cac7b80 msgr2=0x56152cad21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- 172.21.15.33:0/3702482868 shutdown_connections 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- 172.21.15.33:0/3702482868 wait complete. 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 -- start start 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.408+0000 7f219d3c0740 1 Processor -- start 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cadc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cadcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cb3c9f0 con 0x56152cacd260 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cb3cb30 con 0x56152cacdb80 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152cb3ccc0 con 0x56152cad6950 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 10 client.0 _ll_register_callbacks cb 0x7f21880097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56586/0 (socket says 172.21.15.33:56586) 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 -- 172.21.15.33:0/803823295 learned_addr learned my addr 172.21.15.33:0/803823295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f2193fff640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cadc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cadcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 -- 172.21.15.33:0/803823295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 msgr2=0x56152cadcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cadcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 -- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 msgr2=0x56152cadc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cadc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 -- 172.21.15.33:0/803823295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56152ca336d0 con 0x56152cacd260 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21937fe640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2188006010 tx=0x7f2188009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6847affad3c7c11e server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21737fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188018070 con 0x56152cacd260 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2188000e40 con 0x56152cacd260 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.409+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f218800f990 con 0x56152cacd260 2024-01-25T16:20:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f218801e3a0 con 0x56152cacd260 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 10 client.16431 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x56152c9f5a80 con 0x56152cacd260 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2188045080 con 0x56152cacd260 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2188005270 con 0x56152cacd260 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 10 client.16431 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 20 client.16431 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cb3df80 0x56152cb40370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 4 client.16431 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56152c734600 con 0x56152cb3df80 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f2193fff640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cb3df80 0x56152cb40370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f21737fe640 10 client.16431 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.410+0000 7f2193fff640 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cb3df80 0x56152cb40370 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92d230067f579786 server_cookie=32a107d98991489a in_seq=0 out_seq=0 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f21737fe640 1 -- 172.21.15.33:0/803823295 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56152c734600 con 0x56152cb3df80 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f21737fe640 10 client.16431 handle_command_reply: tid=0 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 client.16431 shutdown 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 msgr2=0x56152cadc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2188006010 tx=0x7f2188009f90 comp rx=0 tx=0).stop 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 shutdown_connections 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56152cb3df80 0x56152cb40370 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x56152cad6950 0x56152cadcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56152cacdb80 0x56152cadc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f21737fe640 0 client.16431 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 --2- 172.21.15.33:0/803823295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56152cacd260 0x56152cadc200 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 >> 172.21.15.33:0/803823295 conn(0x56152cac7b80 msgr2=0x56152cac6410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 shutdown_connections 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 1 -- 172.21.15.33:0/803823295 wait complete. 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.700+0000 7f219d3c0740 20 client.16431 trim_cache size 0 max 16384 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:20:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:20:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 msgr2=0x7f217c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 0x7f217c0a3900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f218c0a2250 tx=0x7f218c06c450 comp rx=0 tx=0).stop 2024-01-25T16:20:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 msgr2=0x7f219419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f218c005fd0 tx=0x7f218c00ef10 comp rx=0 tx=0).stop 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 shutdown_connections 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f217c0a1440 0x7f217c0a3900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f219410d430 0x7f219419db10 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2194100230 0x7f21941997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 --2- 172.21.15.33:0/107928238 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f21940ff880 0x7f21941992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 >> 172.21.15.33:0/107928238 conn(0x7f21940fb500 msgr2=0x7f21940fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 shutdown_connections 2024-01-25T16:20:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:19.701+0000 7f219959e640 1 -- 172.21.15.33:0/107928238 wait complete. 2024-01-25T16:20:19.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 892276 8.11 MB/sec execute 556 sec latency 82.129 ms 2024-01-25T16:20:20.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:20 smithi152 ceph-mon[36015]: pgmap v398: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:20.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:20 smithi076 ceph-mon[35642]: pgmap v398: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:20.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 894330 8.12 MB/sec execute 557 sec latency 74.429 ms 2024-01-25T16:20:20.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:20 smithi033 ceph-mon[31417]: pgmap v398: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 255 op/s 2024-01-25T16:20:21.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 895378 8.11 MB/sec execute 558 sec latency 29.016 ms 2024-01-25T16:20:22.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:22 smithi152 ceph-mon[36015]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-01-25T16:20:22.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:22 smithi076 ceph-mon[35642]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-01-25T16:20:22.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 897087 8.11 MB/sec execute 559 sec latency 260.924 ms 2024-01-25T16:20:22.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:22 smithi033 ceph-mon[31417]: pgmap v399: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 363 op/s 2024-01-25T16:20:23.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 898526 8.12 MB/sec execute 560 sec latency 265.398 ms 2024-01-25T16:20:24.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:24 smithi152 ceph-mon[36015]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-01-25T16:20:24.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:24 smithi076 ceph-mon[35642]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-01-25T16:20:24.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 900560 8.12 MB/sec execute 561 sec latency 23.706 ms 2024-01-25T16:20:24.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:24 smithi033 ceph-mon[31417]: pgmap v400: 97 pgs: 97 active+clean; 126 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 223 op/s 2024-01-25T16:20:25.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 902024 8.12 MB/sec execute 562 sec latency 85.860 ms 2024-01-25T16:20:26.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:26 smithi033 ceph-mon[31417]: pgmap v401: 97 pgs: 97 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-01-25T16:20:26.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:26 smithi152 ceph-mon[36015]: pgmap v401: 97 pgs: 97 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-01-25T16:20:26.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:26 smithi076 ceph-mon[35642]: pgmap v401: 97 pgs: 97 active+clean; 135 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.8 MiB/s wr, 303 op/s 2024-01-25T16:20:26.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 903610 8.12 MB/sec execute 563 sec latency 35.998 ms 2024-01-25T16:20:27.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 905507 8.13 MB/sec execute 564 sec latency 41.515 ms 2024-01-25T16:20:28.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:28 smithi152 ceph-mon[36015]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-01-25T16:20:28.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:28 smithi076 ceph-mon[35642]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-01-25T16:20:28.788 INFO:tasks.workunit.client.0.smithi033.stdout: 1 906958 8.12 MB/sec execute 565 sec latency 19.242 ms 2024-01-25T16:20:28.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:28 smithi033 ceph-mon[31417]: pgmap v402: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.7 MiB/s wr, 239 op/s 2024-01-25T16:20:29.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 908962 8.13 MB/sec execute 566 sec latency 26.062 ms 2024-01-25T16:20:30.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:30 smithi152 ceph-mon[36015]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:20:31.005 INFO:tasks.workunit.client.0.smithi033.stdout: 1 910369 8.13 MB/sec execute 567 sec latency 69.835 ms 2024-01-25T16:20:31.006 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:30 smithi076 ceph-mon[35642]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:20:31.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:30 smithi033 ceph-mon[31417]: pgmap v403: 97 pgs: 97 active+clean; 133 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.6 MiB/s wr, 235 op/s 2024-01-25T16:20:31.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 912455 8.14 MB/sec execute 568 sec latency 79.774 ms 2024-01-25T16:20:32.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:32 smithi152 ceph-mon[36015]: pgmap v404: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-01-25T16:20:32.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:32 smithi076 ceph-mon[35642]: pgmap v404: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-01-25T16:20:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- 172.21.15.33:0/4237771533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8106de0 msgr2=0x7f0bf81071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/4237771533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8106de0 0x7f0bf81071c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0bf0009900 tx=0x7f0bf002bcd0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- 172.21.15.33:0/4237771533 shutdown_connections 2024-01-25T16:20:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/4237771533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068d10 0x7f0bf810db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/4237771533 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8068370 0x7f0bf80687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/4237771533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8106de0 0x7f0bf81071c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- 172.21.15.33:0/4237771533 >> 172.21.15.33:0/4237771533 conn(0x7f0bf8076a40 msgr2=0x7f0bf8076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- 172.21.15.33:0/4237771533 shutdown_connections 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- 172.21.15.33:0/4237771533 wait complete. 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 Processor -- start 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 -- start start 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 0x7f0bf8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.787+0000 7f0bfffe3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 0x7f0bf819dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819e1d0 con 0x7f0bf8068d10 2024-01-25T16:20:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819e340 con 0x7f0bf8068370 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf819e500 con 0x7f0bf8106de0 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60282/0 (socket says 172.21.15.33:60282) 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfd57e640 1 -- 172.21.15.33:0/1152258781 learned_addr learned my addr 172.21.15.33:0/1152258781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 0x7f0bf8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfe580640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 0x7f0bf819dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 -- 172.21.15.33:0/1152258781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 msgr2=0x7f0bf819dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 0x7f0bf819dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 -- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 msgr2=0x7f0bf8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 -- 172.21.15.33:0/1152258781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0009560 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfe580640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 0x7f0bf819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 0x7f0bf8199310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0bf002c210 tx=0x7f0bf0005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0beeffd640 1 -- 172.21.15.33:0/1152258781 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf002db40 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf819e780 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0beeffd640 1 -- 172.21.15.33:0/1152258781 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf002dca0 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0beeffd640 1 -- 172.21.15.33:0/1152258781 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf0005150 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.788+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf81a6060 con 0x7f0bf8068370 2024-01-25T16:20:32.792 INFO:tasks.workunit.client.0.smithi033.stdout: 1 913613 8.13 MB/sec execute 569 sec latency 16.124 ms 2024-01-25T16:20:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.791+0000 7f0beeffd640 1 -- 172.21.15.33:0/1152258781 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f0bf00052b0 con 0x7f0bf8068370 2024-01-25T16:20:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.791+0000 7f0c02baf740 1 Processor -- start 2024-01-25T16:20:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 -- start start 2024-01-25T16:20:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43ac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 0x55ecd43aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4312070 con 0x55ecd43a3090 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd430f760 con 0x55ecd43a39b0 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0c02baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd430b050 con 0x55ecd43ac780 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60296/0 (socket says 172.21.15.33:60296) 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfe580640 1 -- 172.21.15.33:0/3913396418 learned_addr learned my addr 172.21.15.33:0/3913396418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 0x55ecd43aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0beeffd640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 0x7f0be40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 -- 172.21.15.33:0/3913396418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 msgr2=0x55ecd43ac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43ac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 -- 172.21.15.33:0/3913396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 msgr2=0x55ecd43a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 -- 172.21.15.33:0/3913396418 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd4302e70 con 0x55ecd43ac780 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfe580640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfe580640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 0x7f0be40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0beeffd640 1 -- 172.21.15.33:0/1152258781 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0bf00ef280 con 0x7f0bf8068370 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.792+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 0x55ecd43aeb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8101e40 tx=0x7f0be80119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b52e6b15205eead server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0be3fff640 1 -- 172.21.15.33:0/3913396418 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800f5c0 con 0x55ecd43ac780 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0be3fff640 1 -- 172.21.15.33:0/3913396418 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be800fbe0 con 0x55ecd43ac780 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0be3fff640 1 -- 172.21.15.33:0/3913396418 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8019cf0 con 0x55ecd43ac780 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0bfe580640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 0x7f0be40a39b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4003f50 tx=0x7f0bf4004a40 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 -- 172.21.15.33:0/3913396418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 msgr2=0x55ecd43aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 0x55ecd43aeb70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8101e40 tx=0x7f0be80119a0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 -- 172.21.15.33:0/3913396418 shutdown_connections 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43ac780 0x55ecd43aeb70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43ac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3913396418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a3090 0x55ecd43a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.793+0000 7f0c02baf740 1 -- 172.21.15.33:0/3913396418 >> 172.21.15.33:0/3913396418 conn(0x55ecd439d9b0 msgr2=0x55ecd439dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- 172.21.15.33:0/3913396418 shutdown_connections 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- 172.21.15.33:0/3913396418 wait complete. 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- start start 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 Processor -- start 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43a9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd42c41e0 con 0x55ecd43ac780 2024-01-25T16:20:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd42c3e80 con 0x55ecd43a39b0 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd42890c0 con 0x55ecd43a3090 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0c02baf740 10 client.0 _ll_register_callbacks cb 0x55ecd42aeac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60308/0 (socket says 172.21.15.33:60308) 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.794+0000 7f0bfe580640 1 -- 172.21.15.33:0/499322308 learned_addr learned my addr 172.21.15.33:0/499322308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 -- 172.21.15.33:0/499322308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 msgr2=0x55ecd43a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 -- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 msgr2=0x55ecd43a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 -- 172.21.15.33:0/499322308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be8009d10 con 0x55ecd43ac780 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.803 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0bfe580640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40049d0 tx=0x7f0bf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=936bd94dcd621d4f server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4004d20 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0bf4005020 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ecd4304040 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bf4004e80 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0c02baf740 10 client.16443 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ecd43096d0 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.795+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf4034020 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0bf4028700 con 0x55ecd43ac780 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 10 client.16443 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 20 client.16443 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd43bc670 0x55ecd43bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 4 client.16443 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ecd4303d60 con 0x55ecd43bc670 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd43bc670 0x55ecd43bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0be17fa640 10 client.16443 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.796+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd43bc670 0x55ecd43bea60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c841f1c621cb9323 server_cookie=8f3962ad4417ad1c in_seq=0 out_seq=0 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.797+0000 7f0be17fa640 1 -- 172.21.15.33:0/499322308 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ecd4303d60 con 0x55ecd43bc670 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.797+0000 7f0be17fa640 10 client.16443 handle_command_reply: tid=0 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.797+0000 7f0c02baf740 1 client.16443 shutdown 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 msgr2=0x55ecd43aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0bf40049d0 tx=0x7f0bf4004ab0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 shutdown_connections 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd43bc670 0x55ecd43bea60 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43aa1c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0be17fa640 0 client.16443 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 --2- 172.21.15.33:0/499322308 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 >> 172.21.15.33:0/499322308 conn(0x55ecd439d9b0 msgr2=0x55ecd439f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 shutdown_connections 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 1 -- 172.21.15.33:0/499322308 wait complete. 2024-01-25T16:20:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.798+0000 7f0c02baf740 20 client.16443 trim_cache size 0 max 16384 2024-01-25T16:20:32.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:32 smithi033 ceph-mon[31417]: pgmap v404: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.9 MiB/s wr, 327 op/s 2024-01-25T16:20:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.811+0000 7f0c02baf740 1 Processor -- start 2024-01-25T16:20:32.810 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.811+0000 7f0c02baf740 1 -- start start 2024-01-25T16:20:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43abe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd4303d60 con 0x55ecd43ac780 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd439e9b0 con 0x55ecd43a39b0 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0c02baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd43b90a0 con 0x55ecd43a3090 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60312/0 (socket says 172.21.15.33:60312) 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 -- 172.21.15.33:0/3944998563 learned_addr learned my addr 172.21.15.33:0/3944998563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 -- 172.21.15.33:0/3944998563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 msgr2=0x55ecd43abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 -- 172.21.15.33:0/3944998563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 msgr2=0x55ecd43a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 -- 172.21.15.33:0/3944998563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecd42a6060 con 0x55ecd43ac780 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0bfe580640 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4011fc0 tx=0x7f0bf4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2581ab3d321c6067 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0be37fe640 1 -- 172.21.15.33:0/3944998563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4003cd0 con 0x55ecd43ac780 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0be37fe640 1 -- 172.21.15.33:0/3944998563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bf4034030 con 0x55ecd43ac780 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.812+0000 7f0be37fe640 1 -- 172.21.15.33:0/3944998563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4038ca0 con 0x55ecd43ac780 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- 172.21.15.33:0/3944998563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 msgr2=0x55ecd43a8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4011fc0 tx=0x7f0bf4004620 comp rx=0 tx=0).stop 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- 172.21.15.33:0/3944998563 shutdown_connections 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43ac780 0x55ecd43a8010 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43a39b0 0x55ecd43a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 --2- 172.21.15.33:0/3944998563 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.818 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- 172.21.15.33:0/3944998563 >> 172.21.15.33:0/3944998563 conn(0x55ecd439d9b0 msgr2=0x55ecd43a4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- 172.21.15.33:0/3944998563 shutdown_connections 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- 172.21.15.33:0/3944998563 wait complete. 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.813+0000 7f0c02baf740 1 -- start start 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 Processor -- start 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 0x55ecd43b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd43b2330 con 0x55ecd43a39b0 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd43b24a0 con 0x55ecd43ac780 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecd43b2660 con 0x55ecd43a3090 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0c02baf740 10 client.0 _ll_register_callbacks cb 0x7f0be8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60314/0 (socket says 172.21.15.33:60314) 2024-01-25T16:20:32.819 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 -- 172.21.15.33:0/2597844284 learned_addr learned my addr 172.21.15.33:0/2597844284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfe580640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 0x55ecd43b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 -- 172.21.15.33:0/2597844284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 msgr2=0x55ecd43b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 -- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 msgr2=0x55ecd43b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 0x55ecd43b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfd57e640 1 -- 172.21.15.33:0/2597844284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecd43096d0 con 0x55ecd43a39b0 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.820 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.814+0000 7f0bfe580640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 0x55ecd43b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0bfd57e640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0be8019860 tx=0x7f0be80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf2868ea4f8df4a server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8011030 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0be8028070 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0be800fbe0 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0be8011190 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 10 client.16449 fetch_fsmap learned FSMap version 41 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55ecd42cba80 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0be800fbe0 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 41) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0be8005440 con 0x55ecd43a39b0 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 10 client.16449 fetch_fsmap finished waiting for FSMap version 41 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 20 client.16449 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:32.821 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd4413f70 0x55ecd4416390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 4 client.16449 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:20:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.815+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ecd400a600 con 0x55ecd4413f70 2024-01-25T16:20:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.816+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd4413f70 0x55ecd4416390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.816+0000 7f0bfdd7f640 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd4413f70 0x55ecd4416390 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4777ae7966e1eb87 server_cookie=74dfc29a4945104a in_seq=0 out_seq=0 2024-01-25T16:20:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.816+0000 7f0be17fa640 10 client.16449 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0be17fa640 1 -- 172.21.15.33:0/2597844284 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ecd400a600 con 0x55ecd4413f70 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0be17fa640 10 client.16449 handle_command_reply: tid=0 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 client.16449 shutdown 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 msgr2=0x55ecd43b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f0be8019860 tx=0x7f0be80197b0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 shutdown_connections 2024-01-25T16:20:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x55ecd4413f70 0x55ecd4416390 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ecd43ac780 0x55ecd43b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ecd43a39b0 0x55ecd43b18b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 --2- 172.21.15.33:0/2597844284 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x55ecd43a3090 0x55ecd43b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0be17fa640 0 client.16449 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 >> 172.21.15.33:0/2597844284 conn(0x55ecd439d9b0 msgr2=0x55ecd43a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 shutdown_connections 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 1 -- 172.21.15.33:0/2597844284 wait complete. 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.941+0000 7f0c02baf740 20 client.16449 trim_cache size 0 max 16384 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:20:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:20:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 msgr2=0x7f0be40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 0x7f0be40a39b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4003f50 tx=0x7f0bf4004a40 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 msgr2=0x7f0bf8199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 0x7f0bf8199310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0bf002c210 tx=0x7f0bf0005d90 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 shutdown_connections 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f0be40a14f0 0x7f0be40a39b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0bf8106de0 0x7f0bf819dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0bf8068d10 0x7f0bf8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 --2- 172.21.15.33:0/1152258781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf8068370 0x7f0bf8199310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 >> 172.21.15.33:0/1152258781 conn(0x7f0bf8076a40 msgr2=0x7f0bf810bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 shutdown_connections 2024-01-25T16:20:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:32.942+0000 7f0bfffe3640 1 -- 172.21.15.33:0/1152258781 wait complete. 2024-01-25T16:20:33.445 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:33 smithi152 ceph-mon[36015]: daemon mds.c restarted 2024-01-25T16:20:33.446 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:20:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:20:33.446 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:20:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:20:33.446 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:20:33 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/731600518 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559e73c03520 con 0x559e73bd8800 2024-01-25T16:20:33.446 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:20:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:20:33.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:33 smithi076 ceph-mon[35642]: daemon mds.c restarted 2024-01-25T16:20:33.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 915879 8.14 MB/sec execute 570 sec latency 60.208 ms 2024-01-25T16:20:33.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:33 smithi033 ceph-mon[31417]: daemon mds.c restarted 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: osdmap e110: 12 total, 12 up, 12 in 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/689332184,v1:172.21.15.152:6837/689332184] up:boot 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:20:34.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:34 smithi152 ceph-mon[36015]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: osdmap e110: 12 total, 12 up, 12 in 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/689332184,v1:172.21.15.152:6837/689332184] up:boot 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:20:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:34 smithi076 ceph-mon[35642]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-01-25T16:20:34.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 916688 8.13 MB/sec execute 571 sec latency 67.487 ms 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: osdmap e110: 12 total, 12 up, 12 in 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/689332184,v1:172.21.15.152:6837/689332184] up:boot 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-01-25T16:20:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:34 smithi033 ceph-mon[31417]: pgmap v406: 97 pgs: 97 active+clean; 138 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 212 op/s 2024-01-25T16:20:35.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 918331 8.16 MB/sec execute 572 sec latency 78.897 ms 2024-01-25T16:20:36.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:36 smithi033 ceph-mon[31417]: pgmap v407: 97 pgs: 97 active+clean; 129 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-01-25T16:20:36.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:36 smithi152 ceph-mon[36015]: pgmap v407: 97 pgs: 97 active+clean; 129 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-01-25T16:20:36.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:36 smithi076 ceph-mon[35642]: pgmap v407: 97 pgs: 97 active+clean; 129 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 270 op/s 2024-01-25T16:20:36.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 920383 8.17 MB/sec execute 573 sec latency 69.416 ms 2024-01-25T16:20:37.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 922162 8.17 MB/sec execute 574 sec latency 16.317 ms 2024-01-25T16:20:38.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:38 smithi152 ceph-mon[36015]: pgmap v408: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:38.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:38 smithi076 ceph-mon[35642]: pgmap v408: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:38.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 924230 8.18 MB/sec execute 575 sec latency 12.995 ms 2024-01-25T16:20:38.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:38 smithi033 ceph-mon[31417]: pgmap v408: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:39.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 926258 8.18 MB/sec execute 576 sec latency 27.925 ms 2024-01-25T16:20:40.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 928074 8.18 MB/sec execute 577 sec latency 19.069 ms 2024-01-25T16:20:41.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:41 smithi152 ceph-mon[36015]: pgmap v409: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:41.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:41 smithi076 ceph-mon[35642]: pgmap v409: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:41.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 929948 8.18 MB/sec execute 578 sec latency 66.190 ms 2024-01-25T16:20:41.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:41 smithi033 ceph-mon[31417]: pgmap v409: 97 pgs: 97 active+clean; 127 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.9 MiB/s wr, 284 op/s 2024-01-25T16:20:42.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 931770 8.19 MB/sec execute 579 sec latency 19.598 ms 2024-01-25T16:20:43.612 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:43 smithi033 ceph-mon[31417]: pgmap v410: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2024-01-25T16:20:43.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:43 smithi152 ceph-mon[36015]: pgmap v410: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2024-01-25T16:20:43.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:43 smithi076 ceph-mon[35642]: pgmap v410: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 285 op/s 2024-01-25T16:20:43.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 934392 8.20 MB/sec execute 580 sec latency 13.336 ms 2024-01-25T16:20:44.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 935408 8.19 MB/sec execute 581 sec latency 167.895 ms 2024-01-25T16:20:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:45 smithi152 ceph-mon[36015]: pgmap v411: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-01-25T16:20:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:45 smithi076 ceph-mon[35642]: pgmap v411: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-01-25T16:20:45.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 937841 8.20 MB/sec execute 582 sec latency 34.813 ms 2024-01-25T16:20:45.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:45 smithi033 ceph-mon[31417]: pgmap v411: 97 pgs: 97 active+clean; 134 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 260 op/s 2024-01-25T16:20:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 -- 172.21.15.33:0/312092495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 msgr2=0x7feb0c10db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 --2- 172.21.15.33:0/312092495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c10db80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7feb04009ca0 tx=0x7feb0402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 -- 172.21.15.33:0/312092495 shutdown_connections 2024-01-25T16:20:46.034 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 --2- 172.21.15.33:0/312092495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c10db80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 --2- 172.21.15.33:0/312092495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 --2- 172.21.15.33:0/312092495 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 0x7feb0c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 -- 172.21.15.33:0/312092495 >> 172.21.15.33:0/312092495 conn(0x7feb0c076a40 msgr2=0x7feb0c076e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 -- 172.21.15.33:0/312092495 shutdown_connections 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.033+0000 7feb11c46640 1 -- 172.21.15.33:0/312092495 wait complete. 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 Processor -- start 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 -- start start 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 0x7feb0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1a5460 con 0x7feb0c068d10 2024-01-25T16:20:46.035 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1a55d0 con 0x7feb0c068370 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb11c46640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1a5790 con 0x7feb0c106e00 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:47094/0 (socket says 172.21.15.33:47094) 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.034+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1563188040 learned_addr learned my addr 172.21.15.33:0/1563188040 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0bfff640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 0x7feb0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1563188040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 msgr2=0x7feb0c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 0x7feb0c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 msgr2=0x7feb0c100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.036 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1563188040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04009900 con 0x7feb0c068370 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0affd640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9f0 tx=0x7feafc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb08ff9640 1 -- 172.21.15.33:0/1563188040 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc009720 con 0x7feb0c068370 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c1a5a70 con 0x7feb0c068370 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb08ff9640 1 -- 172.21.15.33:0/1563188040 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feafc00cd80 con 0x7feb0c068370 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1a5f80 con 0x7feb0c068370 2024-01-25T16:20:46.037 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.035+0000 7feb08ff9640 1 -- 172.21.15.33:0/1563188040 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc018630 con 0x7feb0c068370 2024-01-25T16:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.037+0000 7feb08ff9640 1 -- 172.21.15.33:0/1563188040 <== mon.2 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7feafc004e00 con 0x7feb0c068370 2024-01-25T16:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 Processor -- start 2024-01-25T16:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 -- start start 2024-01-25T16:20:46.044 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb08ff9640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 0x7feaf40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 0x5629675ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675f7780 0x5629675f9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296755d070 con 0x5629675ee9b0 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296755a760 con 0x5629675f7780 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb14808740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x562967556050 con 0x5629675ee090 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0bfff640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 0x7feaf40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb08ff9640 1 -- 172.21.15.33:0/1563188040 <== mon.2 v2:172.21.15.76:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feafc014070 con 0x7feb0c068370 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47738/0 (socket says 172.21.15.33:47738) 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0affd640 1 -- 172.21.15.33:0/1603655718 learned_addr learned my addr 172.21.15.33:0/1603655718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:46.045 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 0x5629675ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0bfff640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675f7780 0x5629675f9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1603655718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675f7780 msgr2=0x5629675f9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675f7780 0x5629675f9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1603655718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 msgr2=0x5629675f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0b7fe640 1 -- 172.21.15.33:0/1603655718 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296754de70 con 0x5629675ee090 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.038+0000 7feb0affd640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feb0bfff640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 0x7feaf40a39b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feb04004af0 tx=0x7feb04032f00 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 0x5629675ee470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feafc0caad0 tx=0x7feafc017fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98790d0f84af1cec server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feaf1ffb640 1 -- 172.21.15.33:0/1603655718 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00f3c0 con 0x5629675ee090 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feaf1ffb640 1 -- 172.21.15.33:0/1603655718 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feafc09a880 con 0x5629675ee090 2024-01-25T16:20:46.046 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feaf1ffb640 1 -- 172.21.15.33:0/1603655718 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0046c0 con 0x5629675ee090 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feb14808740 1 -- 172.21.15.33:0/1603655718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 msgr2=0x5629675ee470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.039+0000 7feb14808740 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 0x5629675ee470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feafc0caad0 tx=0x7feafc017fc0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- 172.21.15.33:0/1603655718 shutdown_connections 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675f7780 0x5629675f9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- 172.21.15.33:0/1603655718 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee090 0x5629675ee470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- 172.21.15.33:0/1603655718 >> 172.21.15.33:0/1603655718 conn(0x5629675e89b0 msgr2=0x5629675e8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- 172.21.15.33:0/1603655718 shutdown_connections 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- 172.21.15.33:0/1603655718 wait complete. 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- start start 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 Processor -- start 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.047 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675f5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296750f1e0 con 0x5629675ee9b0 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296750ee80 con 0x5629675ee090 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.040+0000 7feb14808740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629674d40c0 con 0x5629675f7780 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb14808740 10 client.0 _ll_register_callbacks cb 0x5629674f9ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47754/0 (socket says 172.21.15.33:47754) 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 -- 172.21.15.33:0/333327085 learned_addr learned my addr 172.21.15.33:0/333327085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0bfff640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.048 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 -- 172.21.15.33:0/333327085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 msgr2=0x5629675f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 -- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 msgr2=0x5629675f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 -- 172.21.15.33:0/333327085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1a5f80 con 0x5629675ee9b0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb0affd640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb000129f0 tx=0x7feb00012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ab364b40486755 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb00017070 con 0x5629675ee9b0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feb0000fa40 con 0x5629675ee9b0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56296754f040 con 0x5629675ee9b0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb0000fc40 con 0x5629675ee9b0 2024-01-25T16:20:46.049 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb14808740 10 client.16467 fetch_fsmap learned FSMap version 43 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.041+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5629675546d0 con 0x5629675ee9b0 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56296754f040 con 0x5629675ee9b0 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feb00010430 con 0x5629675ee9b0 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 10 client.16467 fetch_fsmap finished waiting for FSMap version 43 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 20 client.16467 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x562967607630 0x562967609a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 4 client.16467 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56296754ed60 con 0x562967607630 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x562967607630 0x562967609a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x562967607630 0x562967609a70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778909f9adb99320 server_cookie=3ef7fac499020fd6 in_seq=0 out_seq=0 2024-01-25T16:20:46.050 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.042+0000 7feaeaffd640 10 client.16467 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feaeaffd640 1 -- 172.21.15.33:0/333327085 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56296754ed60 con 0x562967607630 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feaeaffd640 10 client.16467 handle_command_reply: tid=0 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 client.16467 shutdown 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 msgr2=0x5629675f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7feb000129f0 tx=0x7feb00012eb0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 shutdown_connections 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x562967607630 0x562967609a70 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee9b0 0x5629675f4ce0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 --2- 172.21.15.33:0/333327085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feaeaffd640 0 client.16467 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:46.051 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.043+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 >> 172.21.15.33:0/333327085 conn(0x5629675e89b0 msgr2=0x5629675ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.044+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 shutdown_connections 2024-01-25T16:20:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.044+0000 7feb14808740 1 -- 172.21.15.33:0/333327085 wait complete. 2024-01-25T16:20:46.052 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.044+0000 7feb14808740 20 client.16467 trim_cache size 0 max 16384 2024-01-25T16:20:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.058+0000 7feb14808740 1 Processor -- start 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.058+0000 7feb14808740 1 -- start start 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.058+0000 7feb14808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb14808740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee9b0 0x5629675fe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb14808740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb14808740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629676040e0 con 0x5629675f7780 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb14808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629675e99b0 con 0x5629675ee090 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb14808740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629675e9580 con 0x5629675ee9b0 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47760/0 (socket says 172.21.15.33:47760) 2024-01-25T16:20:46.062 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0bfff640 1 -- 172.21.15.33:0/4035726768 learned_addr learned my addr 172.21.15.33:0/4035726768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0affd640 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee9b0 0x5629675fe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 -- 172.21.15.33:0/4035726768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee9b0 msgr2=0x5629675fe470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee9b0 0x5629675fe470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 -- 172.21.15.33:0/4035726768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 msgr2=0x5629675fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 -- 172.21.15.33:0/4035726768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629674f1060 con 0x5629675ee090 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0bfff640 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5629675ef040 tx=0x7feafc00cfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a8332b5cd065d2 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feaf17fa640 1 -- 172.21.15.33:0/4035726768 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0b2040 con 0x5629675ee090 2024-01-25T16:20:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feaf17fa640 1 -- 172.21.15.33:0/4035726768 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feafc0a2510 con 0x5629675ee090 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.059+0000 7feaf17fa640 1 -- 172.21.15.33:0/4035726768 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc09e2d0 con 0x5629675ee090 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- 172.21.15.33:0/4035726768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 msgr2=0x5629675f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f5040 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x5629675ef040 tx=0x7feafc00cfa0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- 172.21.15.33:0/4035726768 shutdown_connections 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675f7780 0x5629675fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675ee9b0 0x5629675fe470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 --2- 172.21.15.33:0/4035726768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee090 0x5629675f5040 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- 172.21.15.33:0/4035726768 >> 172.21.15.33:0/4035726768 conn(0x5629675e89b0 msgr2=0x5629675e71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- 172.21.15.33:0/4035726768 shutdown_connections 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- 172.21.15.33:0/4035726768 wait complete. 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.060+0000 7feb14808740 1 -- start start 2024-01-25T16:20:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 Processor -- start 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee9b0 0x5629676584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629675fdcd0 con 0x5629675ee090 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629675fde40 con 0x5629675ee9b0 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629675fe000 con 0x5629675f7780 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb14808740 10 client.0 _ll_register_callbacks cb 0x7feafc00cef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47768/0 (socket says 172.21.15.33:47768) 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 -- 172.21.15.33:0/3605541411 learned_addr learned my addr 172.21.15.33:0/3605541411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0affd640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee9b0 0x5629676584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0bfff640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 -- 172.21.15.33:0/3605541411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 msgr2=0x5629675fd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675fd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 -- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee9b0 msgr2=0x5629676584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee9b0 0x5629676584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 -- 172.21.15.33:0/3605541411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c10e480 con 0x5629675ee090 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feb0b7fe640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feafc096ba0 tx=0x7feafc09a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2dc3ad8e5e12f0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feaeaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc0b2040 con 0x5629675ee090 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.061+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feafc09e2d0 con 0x5629675ee090 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feafc09a880 con 0x5629675ee090 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7feafc0056f0 con 0x5629675ee090 2024-01-25T16:20:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 10 client.16473 fetch_fsmap learned FSMap version 43 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56296754de70 con 0x5629675ee090 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feafc09a880 con 0x5629675ee090 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feafc0a9030 con 0x5629675ee090 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 10 client.16473 fetch_fsmap finished waiting for FSMap version 43 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 20 client.16473 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56296765fa40 0x562967661e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 4 client.16473 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562967255600 con 0x56296765fa40 2024-01-25T16:20:46.067 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.062+0000 7feb0affd640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56296765fa40 0x562967661e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.063+0000 7feaeaffd640 10 client.16473 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.063+0000 7feb0affd640 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56296765fa40 0x562967661e30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa68fe287c6aed32 server_cookie=2fcef17a3763ef9a in_seq=0 out_seq=0 2024-01-25T16:20:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feaeaffd640 1 -- 172.21.15.33:0/3605541411 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562967255600 con 0x56296765fa40 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feaeaffd640 10 client.16473 handle_command_reply: tid=0 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 client.16473 shutdown 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 msgr2=0x562967657f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feafc096ba0 tx=0x7feafc09a9f0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 shutdown_connections 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x56296765fa40 0x562967661e30 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5629675f7780 0x5629675fd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5629675ee9b0 0x5629676584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feaeaffd640 0 client.16473 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 --2- 172.21.15.33:0/3605541411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5629675ee090 0x562967657f60 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 >> 172.21.15.33:0/3605541411 conn(0x5629675e89b0 msgr2=0x5629675e7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 shutdown_connections 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 1 -- 172.21.15.33:0/3605541411 wait complete. 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.254+0000 7feb14808740 20 client.16473 trim_cache size 0 max 16384 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:20:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:20:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 msgr2=0x7feaf40a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 0x7feaf40a39b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7feb04004af0 tx=0x7feb04032f00 comp rx=0 tx=0).stop 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 msgr2=0x7feb0c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e9f0 tx=0x7feafc00edf0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 shutdown_connections 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7feaf40a14f0 0x7feaf40a39b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7feb0c106e00 0x7feb0c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb0c068d10 0x7feb0c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 --2- 172.21.15.33:0/1563188040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feb0c068370 0x7feb0c0fff10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 >> 172.21.15.33:0/1563188040 conn(0x7feb0c076a40 msgr2=0x7feb0c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 shutdown_connections 2024-01-25T16:20:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:46.255+0000 7feb11c46640 1 -- 172.21.15.33:0/1563188040 wait complete. 2024-01-25T16:20:46.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 938956 8.20 MB/sec execute 583 sec latency 119.206 ms 2024-01-25T16:20:47.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:47 smithi152 ceph-mon[36015]: pgmap v412: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-01-25T16:20:47.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:47 smithi076 ceph-mon[35642]: pgmap v412: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-01-25T16:20:47.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 941504 8.21 MB/sec execute 584 sec latency 17.028 ms 2024-01-25T16:20:47.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:47 smithi033 ceph-mon[31417]: pgmap v412: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 380 op/s 2024-01-25T16:20:48.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 943059 8.21 MB/sec execute 585 sec latency 15.506 ms 2024-01-25T16:20:49.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:49 smithi152 ceph-mon[36015]: pgmap v413: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-01-25T16:20:49.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:49 smithi076 ceph-mon[35642]: pgmap v413: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-01-25T16:20:49.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 945167 8.22 MB/sec execute 586 sec latency 83.592 ms 2024-01-25T16:20:49.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:49 smithi033 ceph-mon[31417]: pgmap v413: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.9 MiB/s wr, 263 op/s 2024-01-25T16:20:50.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 946635 8.21 MB/sec execute 587 sec latency 16.507 ms 2024-01-25T16:20:51.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:51 smithi152 ceph-mon[36015]: pgmap v414: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-01-25T16:20:51.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:51 smithi076 ceph-mon[35642]: pgmap v414: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-01-25T16:20:51.790 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948777 8.22 MB/sec execute 588 sec latency 43.255 ms 2024-01-25T16:20:51.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:51 smithi033 ceph-mon[31417]: pgmap v414: 97 pgs: 97 active+clean; 141 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-01-25T16:20:52.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 950356 8.22 MB/sec execute 589 sec latency 20.870 ms 2024-01-25T16:20:53.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:53 smithi152 ceph-mon[36015]: pgmap v415: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2024-01-25T16:20:53.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:53 smithi076 ceph-mon[35642]: pgmap v415: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2024-01-25T16:20:53.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 952587 8.23 MB/sec execute 590 sec latency 14.329 ms 2024-01-25T16:20:53.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:53 smithi033 ceph-mon[31417]: pgmap v415: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 337 op/s 2024-01-25T16:20:54.790 INFO:tasks.workunit.client.0.smithi033.stdout: 1 954555 8.23 MB/sec execute 591 sec latency 106.543 ms 2024-01-25T16:20:55.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:55 smithi152 ceph-mon[36015]: pgmap v416: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-01-25T16:20:55.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:55 smithi076 ceph-mon[35642]: pgmap v416: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-01-25T16:20:55.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 956277 8.23 MB/sec execute 592 sec latency 17.747 ms 2024-01-25T16:20:55.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:55 smithi033 ceph-mon[31417]: pgmap v416: 97 pgs: 97 active+clean; 131 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 245 op/s 2024-01-25T16:20:56.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 958200 8.23 MB/sec execute 593 sec latency 84.192 ms 2024-01-25T16:20:57.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:57 smithi152 ceph-mon[36015]: pgmap v417: 97 pgs: 97 active+clean; 136 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-01-25T16:20:57.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:57 smithi076 ceph-mon[35642]: pgmap v417: 97 pgs: 97 active+clean; 136 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-01-25T16:20:57.789 INFO:tasks.workunit.client.0.smithi033.stdout: 1 960076 8.24 MB/sec execute 594 sec latency 16.605 ms 2024-01-25T16:20:57.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:57 smithi033 ceph-mon[31417]: pgmap v417: 97 pgs: 97 active+clean; 136 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-01-25T16:20:58.790 INFO:tasks.workunit.client.0.smithi033.stdout: 1 962573 8.25 MB/sec execute 595 sec latency 15.448 ms 2024-01-25T16:20:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 -- 172.21.15.33:0/2275777820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 msgr2=0x7fa8ac1060d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/2275777820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac1060d0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009930 tx=0x7fa89c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:20:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 -- 172.21.15.33:0/2275777820 shutdown_connections 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/2275777820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac10dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/2275777820 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 0x7fa8ac0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/2275777820 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac1060d0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 -- 172.21.15.33:0/2275777820 >> 172.21.15.33:0/2275777820 conn(0x7fa8ac0761b0 msgr2=0x7fa8ac0fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 -- 172.21.15.33:0/2275777820 shutdown_connections 2024-01-25T16:20:59.347 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.344+0000 7fa8b1d68640 1 -- 172.21.15.33:0/2275777820 wait complete. 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 Processor -- start 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 -- start start 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 0x7fa8ac197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19c0f0 con 0x7fa8ac105cf0 2024-01-25T16:20:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19c260 con 0x7fa8ac0ffbd0 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8b1d68640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac19c420 con 0x7fa8ac0ff210 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57592/0 (socket says 172.21.15.33:57592) 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 -- 172.21.15.33:0/3417356240 learned_addr learned my addr 172.21.15.33:0/3417356240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.345+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 0x7fa8ac197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 -- 172.21.15.33:0/3417356240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 msgr2=0x7fa8ac197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 0x7fa8ac197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.349 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 -- 172.21.15.33:0/3417356240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 msgr2=0x7fa8ac1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 -- 172.21.15.33:0/3417356240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009560 con 0x7fa8ac105cf0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8abfff640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400eb10 tx=0x7fa8a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8a8ff9640 1 -- 172.21.15.33:0/3417356240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4004b20 con 0x7fa8ac105cf0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8a8ff9640 1 -- 172.21.15.33:0/3417356240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8a4004c80 con 0x7fa8ac105cf0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac19c700 con 0x7fa8ac105cf0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8a8ff9640 1 -- 172.21.15.33:0/3417356240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4010690 con 0x7fa8ac105cf0 2024-01-25T16:20:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.346+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac1a3f80 con 0x7fa8ac105cf0 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8a8ff9640 1 -- 172.21.15.33:0/3417356240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fa8a4005020 con 0x7fa8ac105cf0 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 Processor -- start 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 -- start start 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 0x561bcdba8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdbb1b90 0x561bcdbb3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdb13070 con 0x561bcdbb1b90 2024-01-25T16:20:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdb10760 con 0x561bcdba8dc0 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.349+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdb0c050 con 0x561bcdba84a0 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.33:52250/0 (socket says 172.21.15.33:52250) 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdbb1b90 0x561bcdbb3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 -- 172.21.15.33:0/343634980 learned_addr learned my addr 172.21.15.33:0/343634980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8a8ff9640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 0x7fa8940a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 0x561bcdba8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 -- 172.21.15.33:0/343634980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 msgr2=0x561bcdba8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 0x561bcdba8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 -- 172.21.15.33:0/343634980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdbb1b90 msgr2=0x561bcdbb3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdbb1b90 0x561bcdbb3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 -- 172.21.15.33:0/343634980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcdb03e70 con 0x561bcdba8dc0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 0x561bcdba8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8abfff640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 0x7fa8940a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8a8ff9640 1 -- 172.21.15.33:0/3417356240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa8a4014070 con 0x7fa8ac105cf0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036040 tx=0x7fa89c005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d2117a21ae8d40 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa891ffb640 1 -- 172.21.15.33:0/343634980 <== mon.2 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0046c0 con 0x561bcdba8dc0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa891ffb640 1 -- 172.21.15.33:0/343634980 <== mon.2 v2:172.21.15.76:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa89c004820 con 0x561bcdba8dc0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.350+0000 7fa891ffb640 1 -- 172.21.15.33:0/343634980 <== mon.2 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c005260 con 0x561bcdba8dc0 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 -- 172.21.15.33:0/343634980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 msgr2=0x561bcdbb1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8abfff640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 0x7fa8940a39b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4099f30 tx=0x7fa8a400efd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa89c036040 tx=0x7fa89c005e00 comp rx=0 tx=0).stop 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 -- 172.21.15.33:0/343634980 shutdown_connections 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdbb1b90 0x561bcdbb3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbb1650 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/343634980 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba84a0 0x561bcdba8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 -- 172.21.15.33:0/343634980 >> 172.21.15.33:0/343634980 conn(0x561bcdba2dc0 msgr2=0x561bcdba31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 -- 172.21.15.33:0/343634980 shutdown_connections 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.351+0000 7fa8b4a08740 1 -- 172.21.15.33:0/343634980 wait complete. 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 -- start start 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 Processor -- start 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbaf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdbb1b90 0x561bcdbaf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdac51e0 con 0x561bcdba84a0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdac4e80 con 0x561bcdba8dc0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcda8a0c0 con 0x561bcdbb1b90 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8b4a08740 10 client.0 _ll_register_callbacks cb 0x561bcdaafac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57610/0 (socket says 172.21.15.33:57610) 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 -- 172.21.15.33:0/2581080548 learned_addr learned my addr 172.21.15.33:0/2581080548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbaf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdbb1b90 0x561bcdbaf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.362 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 -- 172.21.15.33:0/2581080548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdbb1b90 msgr2=0x561bcdbaf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.352+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdbb1b90 0x561bcdbaf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8aaffd640 1 -- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 msgr2=0x561bcdbaf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbaf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8aaffd640 1 -- 172.21.15.33:0/2581080548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac10e520 con 0x561bcdba84a0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbaf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005fd0 tx=0x7fa8a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba8f0f4722635fb server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0017040 con 0x561bcdba84a0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa8a000dd60 con 0x561bcdba84a0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bcdb05040 con 0x561bcdba84a0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a0005020 con 0x561bcdba84a0 2024-01-25T16:20:59.363 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 10 client.16485 fetch_fsmap learned FSMap version 43 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561bcdb0a6d0 con 0x561bcdba84a0 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bcdb05040 con 0x561bcdba84a0 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8a0010450 con 0x561bcdba84a0 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 10 client.16485 fetch_fsmap finished waiting for FSMap version 43 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 10 client.16485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 20 client.16485 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdbc1990 0x561bcdbc3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 4 client.16485 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.353+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bcdb04d60 con 0x561bcdbc1990 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.354+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdbc1990 0x561bcdbc3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.354+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdbc1990 0x561bcdbc3db0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e47e53d0b192a2e server_cookie=ebb9a6ee2419ceba in_seq=0 out_seq=0 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.354+0000 7fa88affd640 10 client.16485 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:59.364 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa88affd640 1 -- 172.21.15.33:0/2581080548 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561bcdb04d60 con 0x561bcdbc1990 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa88affd640 10 client.16485 handle_command_reply: tid=0 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 client.16485 shutdown 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 msgr2=0x561bcdbaeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005fd0 tx=0x7fa8a0004730 comp rx=0 tx=0).stop 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 shutdown_connections 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdbc1990 0x561bcdbc3db0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdbb1b90 0x561bcdbaf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdba8dc0 0x561bcdbaf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/2581080548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbaeb00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa88affd640 0 client.16485 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 >> 172.21.15.33:0/2581080548 conn(0x561bcdba2dc0 msgr2=0x561bcdba4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.365 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 shutdown_connections 2024-01-25T16:20:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 1 -- 172.21.15.33:0/2581080548 wait complete. 2024-01-25T16:20:59.366 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.355+0000 7fa8b4a08740 20 client.16485 trim_cache size 0 max 16384 2024-01-25T16:20:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 Processor -- start 2024-01-25T16:20:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 -- start start 2024-01-25T16:20:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.373 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdba2c50 con 0x561bcdba84a0 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdbaff60 con 0x561bcdbb1b90 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdbaa760 con 0x561bcdba8dc0 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57626/0 (socket says 172.21.15.33:57626) 2024-01-25T16:20:59.374 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8aaffd640 1 -- 172.21.15.33:0/3326969701 learned_addr learned my addr 172.21.15.33:0/3326969701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.368+0000 7fa8aaffd640 1 -- 172.21.15.33:0/3326969701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 msgr2=0x561bcdbad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbad3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8aaffd640 1 -- 172.21.15.33:0/3326969701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 msgr2=0x561bcdbad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.375 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8aaffd640 1 -- 172.21.15.33:0/3326969701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bcdaa7060 con 0x561bcdba84a0 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0015010 tx=0x7fa8a0012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9765ec55eb68b9b3 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8917fa640 1 -- 172.21.15.33:0/3326969701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000bc40 con 0x561bcdba84a0 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8917fa640 1 -- 172.21.15.33:0/3326969701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8a0010450 con 0x561bcdba84a0 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8917fa640 1 -- 172.21.15.33:0/3326969701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0005330 con 0x561bcdba84a0 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 -- 172.21.15.33:0/3326969701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 msgr2=0x561bcdbb1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.376 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0015010 tx=0x7fa8a0012730 comp rx=0 tx=0).stop 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 -- 172.21.15.33:0/3326969701 shutdown_connections 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/3326969701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb1230 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.369+0000 7fa8b4a08740 1 -- 172.21.15.33:0/3326969701 >> 172.21.15.33:0/3326969701 conn(0x561bcdba2dc0 msgr2=0x561bcdbc8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- 172.21.15.33:0/3326969701 shutdown_connections 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- 172.21.15.33:0/3326969701 wait complete. 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- start start 2024-01-25T16:20:59.377 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 Processor -- start 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbb7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbb7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdbb7f20 con 0x561bcdba84a0 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdc17ec0 con 0x561bcdbb1b90 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bcdc18000 con 0x561bcdba8dc0 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8b4a08740 10 client.0 _ll_register_callbacks cb 0x7fa89c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8aaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57628/0 (socket says 172.21.15.33:57628) 2024-01-25T16:20:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.370+0000 7fa8aaffd640 1 -- 172.21.15.33:0/694896565 learned_addr learned my addr 172.21.15.33:0/694896565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbb7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbb7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 -- 172.21.15.33:0/694896565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 msgr2=0x561bcdbb7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbb7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 -- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 msgr2=0x561bcdbb7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbb7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 -- 172.21.15.33:0/694896565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bcdb0a6d0 con 0x561bcdba84a0 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8aaffd640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006010 tx=0x7fa8a000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4824442a695ec462 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:20:59.379 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0017030 con 0x561bcdba84a0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa8a0000e40 con 0x561bcdba84a0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa89c004820 con 0x561bcdba84a0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8a000bc40 con 0x561bcdba84a0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 10 client.16491 fetch_fsmap learned FSMap version 43 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x561bcdacca80 con 0x561bcdba84a0 2024-01-25T16:20:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa89c004820 con 0x561bcdba84a0 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 43) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8a0032030 con 0x561bcdba84a0 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 10 client.16491 fetch_fsmap finished waiting for FSMap version 43 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 10 client.16491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 20 client.16491 populate_metadata read hostname 'smithi033' 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdc19360 0x561bcdc1b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 4 client.16491 mds_command: new command op to 24536 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.371+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 --> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bcd80b600 con 0x561bcdc19360 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.372+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdc19360 0x561bcdc1b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:20:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.372+0000 7fa88affd640 10 client.16491 ms_handle_connect on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:59.382 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.372+0000 7fa8ab7fe640 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdc19360 0x561bcdc1b750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca3f8fb6fcf6cb3 server_cookie=a6b81d57e83f3ba4 in_seq=0 out_seq=0 2024-01-25T16:20:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.538+0000 7fa88affd640 1 -- 172.21.15.33:0/694896565 <== mds.0 v2:172.21.15.76:6838/406718297 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bcd80b600 con 0x561bcdc19360 2024-01-25T16:20:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.538+0000 7fa88affd640 10 client.16491 handle_command_reply: tid=0 2024-01-25T16:20:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.538+0000 7fa8b4a08740 1 client.16491 shutdown 2024-01-25T16:20:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 msgr2=0x561bcdbb8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.537 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0006010 tx=0x7fa8a000b600 comp rx=0 tx=0).stop 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 shutdown_connections 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:6838/406718297,v1:172.21.15.76:6839/406718297] conn(0x561bcdc19360 0x561bcdc1b750 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561bcdbb1b90 0x561bcdbb7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x561bcdba8dc0 0x561bcdbb7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa88affd640 0 client.16491 ms_handle_reset on v2:172.21.15.76:6838/406718297 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 --2- 172.21.15.33:0/694896565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561bcdba84a0 0x561bcdbb8d40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 >> 172.21.15.33:0/694896565 conn(0x561bcdba2dc0 msgr2=0x561bcdbad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 shutdown_connections 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 1 -- 172.21.15.33:0/694896565 wait complete. 2024-01-25T16:20:59.538 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.539+0000 7fa8b4a08740 20 client.16491 trim_cache size 0 max 16384 2024-01-25T16:20:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:{ 2024-01-25T16:20:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "message": "", 2024-01-25T16:20:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout: "return_code": 0 2024-01-25T16:20:59.539 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stdout:} 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 msgr2=0x7fa8940a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 0x7fa8940a39b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4099f30 tx=0x7fa8a400efd0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 msgr2=0x7fa8ac19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fa8a400eb10 tx=0x7fa8a400c2d0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 shutdown_connections 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa8940a14f0 0x7fa8940a39b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ac105cf0 0x7fa8ac19baf0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8ac0ffbd0 0x7fa8ac1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 --2- 172.21.15.33:0/3417356240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8ac0ff210 0x7fa8ac197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 >> 172.21.15.33:0/3417356240 conn(0x7fa8ac0761b0 msgr2=0x7fa8ac103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 shutdown_connections 2024-01-25T16:20:59.540 INFO:teuthology.task.background_exec.ubuntu@smithi033.front.sepia.ceph.com.smithi033.stderr:2024-01-25T16:20:59.540+0000 7fa8b1d68640 1 -- 172.21.15.33:0/3417356240 wait complete. 2024-01-25T16:20:59.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:20:59 smithi152 ceph-mon[36015]: pgmap v418: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-01-25T16:20:59.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:20:59 smithi076 ceph-mon[35642]: pgmap v418: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-01-25T16:20:59.790 INFO:tasks.workunit.client.0.smithi033.stdout: 1 964002 8.24 MB/sec execute 596 sec latency 72.947 ms 2024-01-25T16:20:59.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:20:59 smithi033 ceph-mon[31417]: pgmap v418: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 185 op/s 2024-01-25T16:21:00.790 INFO:tasks.workunit.client.0.smithi033.stdout: 1 966086 8.25 MB/sec execute 597 sec latency 28.113 ms 2024-01-25T16:21:01.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:21:01.372 INFO:tasks.workunit.client.0.smithi033.stdout:[966547] write failed on handle 12012 (Resource temporarily unavailable) 2024-01-25T16:21:01.372 INFO:tasks.workunit.client.0.smithi033.stdout:Child failed with status 1 2024-01-25T16:21:01.372 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-01-25T16:21:01.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-25T16:21:01.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:01 smithi152 ceph-mon[36015]: pgmap v419: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-01-25T16:21:01.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:01 smithi076 ceph-mon[35642]: pgmap v419: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-01-25T16:21:01.689 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:01 smithi033 ceph-mon[31417]: pgmap v419: 97 pgs: 97 active+clean; 135 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-01-25T16:21:01.690 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-01-25T16:21:01.948 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=80c39e98449a4b6eaae28d70be38461b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_661946e7044723130c03627323f6a879a9848cb5/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c856e0f9244dd29a50697123b480d63f16a4ed81/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-01-25T16:21:01.952 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-01-25T16:21:02.582 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:21:02.597 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-01-25T16:21:02.609 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-01-25T16:21:02.609 INFO:teuthology.task.exec:Executing custom commands... 2024-01-25T16:21:02.609 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:21:02.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-01-25T16:21:02.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-01-25T16:21:02.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-01-25T16:21:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/1183222887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 msgr2=0x7f4aa010d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/1183222887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa010d590 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f4a9401a910 tx=0x7f4a9403cca0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/1183222887 shutdown_connections 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/1183222887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa011c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/1183222887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa010d590 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/1183222887 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa010d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/1183222887 >> 172.21.15.33:0/1183222887 conn(0x7f4aa006bd90 msgr2=0x7f4aa006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/1183222887 shutdown_connections 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.909+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/1183222887 wait complete. 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.910+0000 7f4aa5f5a640 1 Processor -- start 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.910+0000 7f4aa5f5a640 1 -- start start 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.910+0000 7f4aa5f5a640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa01199e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:02.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4aa5f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa0119f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4aa5f5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa0115ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4aa5f5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0116430 con 0x7f4aa010f8a0 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4aa5f5a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa01165a0 con 0x7f4aa011a4d0 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4aa5f5a640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa0116760 con 0x7f4aa010f430 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4a9f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa01199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4a9f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa01199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.33:41742/0 (socket says 172.21.15.33:41742) 2024-01-25T16:21:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4a9f7fe640 1 -- 172.21.15.33:0/2050709746 learned_addr learned my addr 172.21.15.33:0/2050709746 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4a9ffff640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa0115ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.911+0000 7f4a9effd640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa0119f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.912+0000 7f4a9effd640 1 -- 172.21.15.33:0/2050709746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 msgr2=0x7f4aa01199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.912+0000 7f4a9effd640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa01199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.912+0000 7f4a9effd640 1 -- 172.21.15.33:0/2050709746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 msgr2=0x7f4aa0115ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.912+0000 7f4a9effd640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa0115ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.912+0000 7f4a9effd640 1 -- 172.21.15.33:0/2050709746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9401a570 con 0x7f4aa010f8a0 2024-01-25T16:21:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.913+0000 7f4a9ffff640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa0115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-25T16:21:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.913+0000 7f4a9effd640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa0119f20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4a9401a860 tx=0x7f4a940045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.913+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94005150 con 0x7f4aa010f8a0 2024-01-25T16:21:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.913+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a940035f0 con 0x7f4aa010f8a0 2024-01-25T16:21:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.913+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a94045440 con 0x7f4aa010f8a0 2024-01-25T16:21:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.914+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa01aa160 con 0x7f4aa010f8a0 2024-01-25T16:21:02.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.914+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa01aa4d0 con 0x7f4aa010f8a0 2024-01-25T16:21:02.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.919+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4a940052b0 con 0x7f4aa010f8a0 2024-01-25T16:21:02.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.919+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa006d030 con 0x7f4aa010f8a0 2024-01-25T16:21:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.919+0000 7f4a9cff9640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 0x7f4a880a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.920+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4a940ffed0 con 0x7f4aa010f8a0 2024-01-25T16:21:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.923+0000 7f4a9f7fe640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 0x7f4a880a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:02.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.924+0000 7f4a9f7fe640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 0x7f4a880a3b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009a30 tx=0x7f4a90009190 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:02.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:02.928+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4a940c9050 con 0x7f4aa010f8a0 2024-01-25T16:21:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.317+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4aa0117040 con 0x7f4a880a1640 2024-01-25T16:21:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.332+0000 7f4a9cff9640 1 -- 172.21.15.33:0/2050709746 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f4aa0117040 con 0x7f4a880a1640 2024-01-25T16:21:03.330 INFO:teuthology.orchestra.run.smithi033.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-01-25T00:00:00", "created": "2024-01-25T16:07:51", "first": "2024-01-25T16:08:00", "last": "2024-01-25T16:21:00", "last_pruned": "2024-01-25T16:21:00", "created_count": 14, "pruned_count": 8, "active": true} 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 msgr2=0x7f4a880a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 0x7f4a880a3b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009a30 tx=0x7f4a90009190 comp rx=0 tx=0).stop 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 msgr2=0x7f4aa0119f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa0119f20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4a9401a860 tx=0x7f4a940045e0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 shutdown_connections 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4a880a1640 0x7f4a880a3b00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4aa011a4d0 0x7f4aa0115ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4aa010f8a0 0x7f4aa0119f20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 --2- 172.21.15.33:0/2050709746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4aa010f430 0x7f4aa01199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 >> 172.21.15.33:0/2050709746 conn(0x7f4aa006bd90 msgr2=0x7f4aa011b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 shutdown_connections 2024-01-25T16:21:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.335+0000 7f4aa5f5a640 1 -- 172.21.15.33:0/2050709746 wait complete. 2024-01-25T16:21:03.350 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-01-25T16:21:03.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 -- 172.21.15.33:0/3790188058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 msgr2=0x7f3180109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 --2- 172.21.15.33:0/3790188058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f3180109a70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3170009900 tx=0x7f317002bd90 comp rx=0 tx=0).stop 2024-01-25T16:21:03.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 -- 172.21.15.33:0/3790188058 shutdown_connections 2024-01-25T16:21:03.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 --2- 172.21.15.33:0/3790188058 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180112c40 0x7f3180115030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 --2- 172.21.15.33:0/3790188058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180109fb0 0x7f3180112700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 --2- 172.21.15.33:0/3790188058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f3180109a70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.527+0000 7f318587c640 1 -- 172.21.15.33:0/3790188058 >> 172.21.15.33:0/3790188058 conn(0x7f318006bd90 msgr2=0x7f318006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:03.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.528+0000 7f318587c640 1 -- 172.21.15.33:0/3790188058 shutdown_connections 2024-01-25T16:21:03.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.529+0000 7f318587c640 1 -- 172.21.15.33:0/3790188058 wait complete. 2024-01-25T16:21:03.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.529+0000 7f318587c640 1 Processor -- start 2024-01-25T16:21:03.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 -- start start 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f31801abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180109fb0 0x7f31801ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31801a69b0 con 0x7f3180112c40 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31801a6b20 con 0x7f3180109690 2024-01-25T16:21:03.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f318587c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31801a6ce0 con 0x7f3180109fb0 2024-01-25T16:21:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f317f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57658/0 (socket says 172.21.15.33:57658) 2024-01-25T16:21:03.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.530+0000 7f317f7fe640 1 -- 172.21.15.33:0/978436568 learned_addr learned my addr 172.21.15.33:0/978436568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317effd640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f31801abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317f7fe640 1 -- 172.21.15.33:0/978436568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180109fb0 msgr2=0x7f31801ac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317f7fe640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180109fb0 0x7f31801ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317f7fe640 1 -- 172.21.15.33:0/978436568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 msgr2=0x7f31801abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317f7fe640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f31801abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.531+0000 7f317f7fe640 1 -- 172.21.15.33:0/978436568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3170009560 con 0x7f3180112c40 2024-01-25T16:21:03.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f317f7fe640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f317800b990 tx=0x7f317800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317800cba0 con 0x7f3180112c40 2024-01-25T16:21:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31801a6fc0 con 0x7f3180112c40 2024-01-25T16:21:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31801b2dd0 con 0x7f3180112c40 2024-01-25T16:21:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f317800cd00 con 0x7f3180112c40 2024-01-25T16:21:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.532+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178007770 con 0x7f3180112c40 2024-01-25T16:21:03.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.536+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f31780078d0 con 0x7f3180112c40 2024-01-25T16:21:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.536+0000 7f318487a640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 0x7f31680a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.537+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f31780cd6b0 con 0x7f3180112c40 2024-01-25T16:21:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.537+0000 7f317effd640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 0x7f31680a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.538+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c005290 con 0x7f3180112c40 2024-01-25T16:21:03.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.540+0000 7f317effd640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 0x7f31680a3be0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3170000c00 tx=0x7f3170005fd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:03.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.547+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f317804f600 con 0x7f3180112c40 2024-01-25T16:21:03.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:03 smithi152 ceph-mon[36015]: pgmap v420: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-01-25T16:21:03.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:03 smithi152 ceph-mon[36015]: osdmap e111: 12 total, 12 up, 12 in 2024-01-25T16:21:03.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:03 smithi076 ceph-mon[35642]: pgmap v420: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-01-25T16:21:03.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:03 smithi076 ceph-mon[35642]: osdmap e111: 12 total, 12 up, 12 in 2024-01-25T16:21:03.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:03 smithi033 ceph-mon[31417]: pgmap v420: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.3 MiB/s wr, 373 op/s 2024-01-25T16:21:03.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:03 smithi033 ceph-mon[31417]: osdmap e111: 12 total, 12 up, 12 in 2024-01-25T16:21:03.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.920+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 --> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f316c000bc0 con 0x7f31680a1720 2024-01-25T16:21:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.929+0000 7f318487a640 1 -- 172.21.15.33:0/978436568 <== mgr.15318 v2:172.21.15.33:6800/2840995268 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f316c000bc0 con 0x7f31680a1720 2024-01-25T16:21:03.927 INFO:teuthology.orchestra.run.smithi033.stdout:/ 1m 6m3h 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.930+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 msgr2=0x7f31680a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.930+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 0x7f31680a3be0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3170000c00 tx=0x7f3170005fd0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.930+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 msgr2=0x7f31801a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.930+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f317800b990 tx=0x7f317800be50 comp rx=0 tx=0).stop 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 shutdown_connections 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f31680a1720 0x7f31680a3be0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3180112c40 0x7f31801a6360 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3180109fb0 0x7f31801ac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 --2- 172.21.15.33:0/978436568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3180109690 0x7f31801abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 >> 172.21.15.33:0/978436568 conn(0x7f318006bd90 msgr2=0x7f3180108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 shutdown_connections 2024-01-25T16:21:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:03.931+0000 7f318587c640 1 -- 172.21.15.33:0/978436568 wait complete. 2024-01-25T16:21:03.939 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-01-25T16:21:03.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:21:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 -- 172.21.15.33:0/3544780446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4100890 msgr2=0x7f29b4104880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 --2- 172.21.15.33:0/3544780446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4100890 0x7f29b4104880 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009930 tx=0x7f29a402bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 -- 172.21.15.33:0/3544780446 shutdown_connections 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 --2- 172.21.15.33:0/3544780446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b4104f60 0x7f29b410cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 --2- 172.21.15.33:0/3544780446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4100890 0x7f29b4104880 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 --2- 172.21.15.33:0/3544780446 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 0x7f29b41002c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 -- 172.21.15.33:0/3544780446 >> 172.21.15.33:0/3544780446 conn(0x7f29b4076d50 msgr2=0x7f29b4077160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 -- 172.21.15.33:0/3544780446 shutdown_connections 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 -- 172.21.15.33:0/3544780446 wait complete. 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.064+0000 7f29bc56e640 1 Processor -- start 2024-01-25T16:21:04.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 -- start start 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 0x7f29b4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b419bbe0 0x7f29b419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b419c5d0 con 0x7f29b4104f60 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b419c740 con 0x7f29b419bbe0 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b419c8b0 con 0x7f29b40ffee0 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57686/0 (socket says 172.21.15.33:57686) 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 -- 172.21.15.33:0/4124883227 learned_addr learned my addr 172.21.15.33:0/4124883227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bab0b640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b419bbe0 0x7f29b419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29ba30a640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 0x7f29b4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 -- 172.21.15.33:0/4124883227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 msgr2=0x7f29b4197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 0x7f29b4197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 -- 172.21.15.33:0/4124883227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b419bbe0 msgr2=0x7f29b419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b419bbe0 0x7f29b419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 -- 172.21.15.33:0/4124883227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a4009560 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29b9b09640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009230 tx=0x7f29a402cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4004310 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29a4004470 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.065+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b41a3e90 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.066+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4003d80 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.066+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b41a4250 con 0x7f29b4104f60 2024-01-25T16:21:04.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.067+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29b4101360 con 0x7f29b4104f60 2024-01-25T16:21:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.067+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f29a4005000 con 0x7f29b4104f60 2024-01-25T16:21:04.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.068+0000 7f29ab7fe640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 0x7f29a00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.068+0000 7f29ba30a640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 0x7f29a00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.068+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f29a40ef710 con 0x7f29b4104f60 2024-01-25T16:21:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.068+0000 7f29ba30a640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 0x7f29a00a3910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009b60 tx=0x7f29b0005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.072+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f29a40b88f0 con 0x7f29b4104f60 2024-01-25T16:21:04.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.253+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29b4198670 con 0x7f29b4104f60 2024-01-25T16:21:04.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.255+0000 7f29ab7fe640 1 -- 172.21.15.33:0/4124883227 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+23046 (secure 0 0 0) 0x7f29a40bbf10 con 0x7f29b4104f60 2024-01-25T16:21:04.253 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:04.253 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":111,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:21:02.337960+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1587480808":"2024-01-25T17:12:43.374402+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2511336685":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/69013124":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/679037113":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/1063041969":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/441508626":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1063041969":"2024-01-26T16:07:50.056976+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 msgr2=0x7f29a00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 0x7f29a00a3910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009b60 tx=0x7f29b0005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 msgr2=0x7f29b4197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009230 tx=0x7f29a402cd20 comp rx=0 tx=0).stop 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 shutdown_connections 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f29a00a1450 0x7f29a00a3910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b419bbe0 0x7f29b419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f29b4104f60 0x7f29b4197860 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 --2- 172.21.15.33:0/4124883227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f29b40ffee0 0x7f29b4197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 >> 172.21.15.33:0/4124883227 conn(0x7f29b4076d50 msgr2=0x7f29b40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 shutdown_connections 2024-01-25T16:21:04.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.257+0000 7f29bc56e640 1 -- 172.21.15.33:0/4124883227 wait complete. 2024-01-25T16:21:04.266 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:21:04.267 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:21:04.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.375+0000 7f6446382640 1 -- 172.21.15.33:0/1944765753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64400ffdd0 msgr2=0x7f64401001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.375+0000 7f6446382640 1 --2- 172.21.15.33:0/1944765753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64400ffdd0 0x7f64401001b0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6430009930 tx=0x7f643002bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 -- 172.21.15.33:0/1944765753 shutdown_connections 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- 172.21.15.33:0/1944765753 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6440104ea0 0x7f644010ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- 172.21.15.33:0/1944765753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440100780 0x7f64401047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- 172.21.15.33:0/1944765753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64400ffdd0 0x7f64401001b0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 -- 172.21.15.33:0/1944765753 >> 172.21.15.33:0/1944765753 conn(0x7f6440076cf0 msgr2=0x7f6440077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 -- 172.21.15.33:0/1944765753 shutdown_connections 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 -- 172.21.15.33:0/1944765753 wait complete. 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 Processor -- start 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 -- start start 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64400ffdd0 0x7f64401980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.376+0000 7f6446382640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 0x7f64401a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f6446382640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a3810 con 0x7f6440104ea0 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f6446382640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a3980 con 0x7f6440198b90 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f6446382640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64401a3b40 con 0x7f64400ffdd0 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57708/0 (socket says 172.21.15.33:57708) 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 -- 172.21.15.33:0/263314667 learned_addr learned my addr 172.21.15.33:0/263314667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643ffff640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64400ffdd0 0x7f64401980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f644491f640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 0x7f64401a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 -- 172.21.15.33:0/263314667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64400ffdd0 msgr2=0x7f64401980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64400ffdd0 0x7f64401980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 -- 172.21.15.33:0/263314667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 msgr2=0x7f64401a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 0x7f64401a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 -- 172.21.15.33:0/263314667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009560 con 0x7f6440104ea0 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f644491f640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 0x7f64401a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643f7fe640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f643400d770 tx=0x7f643400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434009c30 con 0x7f6440104ea0 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64401a3e20 con 0x7f6440104ea0 2024-01-25T16:21:04.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643400de80 con 0x7f6440104ea0 2024-01-25T16:21:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400f620 con 0x7f6440104ea0 2024-01-25T16:21:04.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.377+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64401a42a0 con 0x7f6440104ea0 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.378+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6440067ae0 con 0x7f6440104ea0 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.380+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6434005030 con 0x7f6440104ea0 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.380+0000 7f643d7fa640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 0x7f64280a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.380+0000 7f643ffff640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 0x7f64280a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.380+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f64340ccda0 con 0x7f6440104ea0 2024-01-25T16:21:04.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.381+0000 7f643ffff640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 0x7f64280a3910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6430005ab0 tx=0x7f6430005fd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.385+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6434095e70 con 0x7f6440104ea0 2024-01-25T16:21:04.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:04 smithi033 ceph-mon[31417]: from='client.16497 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:04 smithi033 ceph-mon[31417]: from='client.16500 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:04 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4124883227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.564+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6440100780 con 0x7f6440104ea0 2024-01-25T16:21:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.565+0000 7f643d7fa640 1 -- 172.21.15.33:0/263314667 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64340210a0 con 0x7f6440104ea0 2024-01-25T16:21:04.562 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:21:04.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.566+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 msgr2=0x7f64280a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.566+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 0x7f64280a3910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6430005ab0 tx=0x7f6430005fd0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.566+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 msgr2=0x7f6440198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.566+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f643400d770 tx=0x7f643400dc30 comp rx=0 tx=0).stop 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 shutdown_connections 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f64280a1450 0x7f64280a3910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6440198b90 0x7f64401a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6440104ea0 0x7f6440198600 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 --2- 172.21.15.33:0/263314667 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f64400ffdd0 0x7f64401980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 >> 172.21.15.33:0/263314667 conn(0x7f6440076cf0 msgr2=0x7f64400fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 shutdown_connections 2024-01-25T16:21:04.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.567+0000 7f6446382640 1 -- 172.21.15.33:0/263314667 wait complete. 2024-01-25T16:21:04.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:21:04.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:04 smithi152 ceph-mon[36015]: from='client.16497 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:04 smithi152 ceph-mon[36015]: from='client.16500 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:04 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4124883227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:04.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:04 smithi076 ceph-mon[35642]: from='client.16497 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:04 smithi076 ceph-mon[35642]: from='client.16500 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-01-25T16:21:04.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:04 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4124883227' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 -- 172.21.15.33:0/3492346749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb041018a0 msgr2=0x7fbb041058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/3492346749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb041018a0 0x7fbb041058e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8009930 tx=0x7fbaf802bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 -- 172.21.15.33:0/3492346749 shutdown_connections 2024-01-25T16:21:04.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/3492346749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 0x7fbb0410db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/3492346749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb041018a0 0x7fbb041058e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/3492346749 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb04100ef0 0x7fbb041012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 -- 172.21.15.33:0/3492346749 >> 172.21.15.33:0/3492346749 conn(0x7fbb040780f0 msgr2=0x7fbb04078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 -- 172.21.15.33:0/3492346749 shutdown_connections 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.683+0000 7fbb0a85c640 1 -- 172.21.15.33:0/3492346749 wait complete. 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 Processor -- start 2024-01-25T16:21:04.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 -- start start 2024-01-25T16:21:04.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 0x7fbb04197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0419bb00 0x7fbb0419bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0419c4f0 con 0x7fbb04100ef0 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0419c660 con 0x7fbb04105fc0 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb0a85c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0419c7d0 con 0x7fbb0419bb00 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57726/0 (socket says 172.21.15.33:57726) 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 -- 172.21.15.33:0/2541038570 learned_addr learned my addr 172.21.15.33:0/2541038570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:04.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb037fe640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 0x7fbb04197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb08df9640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0419bb00 0x7fbb0419bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 -- 172.21.15.33:0/2541038570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0419bb00 msgr2=0x7fbb0419bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0419bb00 0x7fbb0419bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 -- 172.21.15.33:0/2541038570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 msgr2=0x7fbb04197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 0x7fbb04197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.684+0000 7fbb03fff640 1 -- 172.21.15.33:0/2541038570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf8009560 con 0x7fbb04100ef0 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb03fff640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c890 tx=0x7fbaf400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004220 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb041a3db0 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf4007e70 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4005690 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.685+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb041a41d0 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.686+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb04067f30 con 0x7fbb04100ef0 2024-01-25T16:21:04.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.687+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fbaf40079b0 con 0x7fbb04100ef0 2024-01-25T16:21:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.687+0000 7fbb017fa640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 0x7fbaec0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.687+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbaf40ccc20 con 0x7fbb04100ef0 2024-01-25T16:21:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.687+0000 7fbb037fe640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 0x7fbaec0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.688+0000 7fbb037fe640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 0x7fbaec0a3910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8000c00 tx=0x7fbaf8005fd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.692+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fbaf4095da0 con 0x7fbb04100ef0 2024-01-25T16:21:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.871+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb041018a0 con 0x7fbb04100ef0 2024-01-25T16:21:04.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.871+0000 7fbb017fa640 1 -- 172.21.15.33:0/2541038570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbaf40993c0 con 0x7fbb04100ef0 2024-01-25T16:21:04.869 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:21:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 msgr2=0x7fbaec0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 0x7fbaec0a3910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8000c00 tx=0x7fbaf8005fd0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 msgr2=0x7fbb04197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbaf400c890 tx=0x7fbaf400cd50 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 shutdown_connections 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fbaec0a1450 0x7fbaec0a3910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb0419bb00 0x7fbb0419bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb04105fc0 0x7fbb04197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 --2- 172.21.15.33:0/2541038570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb04100ef0 0x7fbb04197240 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.873+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 >> 172.21.15.33:0/2541038570 conn(0x7fbb040780f0 msgr2=0x7fbb040ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.874+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 shutdown_connections 2024-01-25T16:21:04.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.874+0000 7fbb0a85c640 1 -- 172.21.15.33:0/2541038570 wait complete. 2024-01-25T16:21:04.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:21:04.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 -- 172.21.15.33:0/1133003189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1018b0 msgr2=0x7fa80c1058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 --2- 172.21.15.33:0/1133003189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1018b0 0x7fa80c1058f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009930 tx=0x7fa7fc02bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 -- 172.21.15.33:0/1133003189 shutdown_connections 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 --2- 172.21.15.33:0/1133003189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 0x7fa80c10dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 --2- 172.21.15.33:0/1133003189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c1018b0 0x7fa80c1058f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 --2- 172.21.15.33:0/1133003189 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c100f00 0x7fa80c1012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 -- 172.21.15.33:0/1133003189 >> 172.21.15.33:0/1133003189 conn(0x7fa80c0fc720 msgr2=0x7fa80c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:04.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 -- 172.21.15.33:0/1133003189 shutdown_connections 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.990+0000 7fa81417d640 1 -- 172.21.15.33:0/1133003189 wait complete. 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 Processor -- start 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 -- start start 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 0x7fa80c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c19bb50 0x7fa80c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19c540 con 0x7fa80c100f00 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19c6b0 con 0x7fa80c105fd0 2024-01-25T16:21:04.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81417d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c19c820 con 0x7fa80c19bb50 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa811f19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa811f19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57750/0 (socket says 172.21.15.33:57750) 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa811f19640 1 -- 172.21.15.33:0/1968683744 learned_addr learned my addr 172.21.15.33:0/1968683744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa811718640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 0x7fa80c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.991+0000 7fa81271a640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c19bb50 0x7fa80c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 -- 172.21.15.33:0/1968683744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c19bb50 msgr2=0x7fa80c19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c19bb50 0x7fa80c19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 -- 172.21.15.33:0/1968683744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 msgr2=0x7fa80c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 0x7fa80c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 -- 172.21.15.33:0/1968683744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc009560 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa811f19640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d770 tx=0x7fa80800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808009c30 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c1a3e00 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa80800de80 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa808004de0 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.992+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c1a41d0 con 0x7fa80c100f00 2024-01-25T16:21:04.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.993+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c067f30 con 0x7fa80c100f00 2024-01-25T16:21:04.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.994+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fa808004f40 con 0x7fa80c100f00 2024-01-25T16:21:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.995+0000 7fa802ffd640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 0x7fa7f80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.995+0000 7fa811718640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 0x7fa7f80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.995+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa8080ccd60 con 0x7fa80c100f00 2024-01-25T16:21:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.995+0000 7fa811718640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 0x7fa7f80a3910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc005fd0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:04.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:04.998+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fa808095ee0 con 0x7fa80c100f00 2024-01-25T16:21:05.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.180+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa80c1018b0 con 0x7fa80c100f00 2024-01-25T16:21:05.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.181+0000 7fa802ffd640 1 -- 172.21.15.33:0/1968683744 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8080170a0 con 0x7fa80c100f00 2024-01-25T16:21:05.179 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:21:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 msgr2=0x7fa7f80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 0x7fa7f80a3910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc000c00 tx=0x7fa7fc005fd0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 msgr2=0x7fa80c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:05.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa80800d770 tx=0x7fa80800dc30 comp rx=0 tx=0).stop 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 shutdown_connections 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fa7f80a1450 0x7fa7f80a3910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa80c19bb50 0x7fa80c19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa80c105fd0 0x7fa80c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 --2- 172.21.15.33:0/1968683744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa80c100f00 0x7fa80c197290 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 >> 172.21.15.33:0/1968683744 conn(0x7fa80c0fc720 msgr2=0x7fa80c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 shutdown_connections 2024-01-25T16:21:05.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.183+0000 7fa81417d640 1 -- 172.21.15.33:0/1968683744 wait complete. 2024-01-25T16:21:05.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-01-25T16:21:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 -- 172.21.15.33:0/3067938808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc104fc0 msgr2=0x7f13dc10cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 --2- 172.21.15.33:0/3067938808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc104fc0 0x7f13dc10cb00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009cd0 tx=0x7f13d402c0f0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 -- 172.21.15.33:0/3067938808 shutdown_connections 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 --2- 172.21.15.33:0/3067938808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc104fc0 0x7f13dc10cb00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 --2- 172.21.15.33:0/3067938808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc1012f0 0x7f13dc104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 --2- 172.21.15.33:0/3067938808 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc100940 0x7f13dc100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 -- 172.21.15.33:0/3067938808 >> 172.21.15.33:0/3067938808 conn(0x7f13dc074f40 msgr2=0x7f13dc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 -- 172.21.15.33:0/3067938808 shutdown_connections 2024-01-25T16:21:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.298+0000 7f13e1b25640 1 -- 172.21.15.33:0/3067938808 wait complete. 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 Processor -- start 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 -- start start 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 0x7f13dc101e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 0x7f13dc1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a5980 con 0x7f13dc100940 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a5af0 con 0x7f13dc104fc0 2024-01-25T16:21:05.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13e1b25640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1a5cb0 con 0x7f13dc1023b0 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13db7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57756/0 (socket says 172.21.15.33:57756) 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13db7fe640 1 -- 172.21.15.33:0/342322535 learned_addr learned my addr 172.21.15.33:0/342322535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13daffd640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 0x7f13dc101e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13dbfff640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 0x7f13dc1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.299+0000 7f13db7fe640 1 -- 172.21.15.33:0/342322535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 msgr2=0x7f13dc1a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13db7fe640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 0x7f13dc1a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13db7fe640 1 -- 172.21.15.33:0/342322535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 msgr2=0x7f13dc101e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13db7fe640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 0x7f13dc101e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13db7fe640 1 -- 172.21.15.33:0/342322535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009900 con 0x7f13dc100940 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13daffd640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 0x7f13dc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13dbfff640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 0x7f13dc1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13db7fe640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc009720 con 0x7f13dc100940 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc1a5f90 con 0x7f13dc100940 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f13cc00cd80 con 0x7f13dc100940 2024-01-25T16:21:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc010430 con 0x7f13dc100940 2024-01-25T16:21:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.300+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1a63c0 con 0x7f13dc100940 2024-01-25T16:21:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.301+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13dc067f30 con 0x7f13dc100940 2024-01-25T16:21:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.303+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f13cc005000 con 0x7f13dc100940 2024-01-25T16:21:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.303+0000 7f13d8ff9640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 0x7f13c40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.303+0000 7f13daffd640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 0x7f13c40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.303+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f13cc014070 con 0x7f13dc100940 2024-01-25T16:21:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.304+0000 7f13daffd640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 0x7f13c40a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13dc1033c0 tx=0x7f13d0005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.307+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f13cc0966d0 con 0x7f13dc100940 2024-01-25T16:21:05.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.541+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f13dc1012f0 con 0x7f13dc100940 2024-01-25T16:21:05.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.549+0000 7f13d8ff9640 1 -- 172.21.15.33:0/342322535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) v1 ==== 81+0+732995 (secure 0 0 0) 0x7f13cc099cf0 con 0x7f13dc100940 2024-01-25T16:21:05.557 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:05.557 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:21:05.557 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 26, 2024-01-25T16:21:05.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.557 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 15318, 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2840995268 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2840995268 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/2840995268", 2024-01-25T16:21:05.559 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-01-25T16:07:50.057274+0000", 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540701272860131327, 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24929, 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701272860131327, 2024-01-25T16:21:05.560 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.564 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.565 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.568 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-01-25T16:21:05.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-01-25T16:21:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-01-25T16:21:05.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.580 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-01-25T16:21:05.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-25T16:21:05.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.585 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.586 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:05.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-01-25T16:21:05.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-25T16:21:05.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-01-25T16:21:05.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-01-25T16:21:05.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-01-25T16:21:05.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.601 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.605 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.606 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:05.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-01-25T16:21:05.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-01-25T16:21:05.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-01-25T16:21:05.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-25T16:21:05.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.618 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-01-25T16:21:05.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-25T16:21:05.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-01-25T16:21:05.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-01-25T16:21:05.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-01-25T16:21:05.630 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-01-25T16:21:05.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-01-25T16:21:05.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-25T16:21:05.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-01-25T16:21:05.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.641 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-01-25T16:21:05.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-01-25T16:21:05.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-01-25T16:21:05.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.651 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-25T16:21:05.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.656 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-01-25T16:21:05.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-01-25T16:21:05.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-01-25T16:21:05.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-01-25T16:21:05.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:05.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.674 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.677 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.683 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.692 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-01-25T16:21:05.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-01-25T16:21:05.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-25T16:21:05.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-01-25T16:21:05.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-25T16:21:05.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-01-25T16:21:05.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.717 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.719 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.722 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.723 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-01-25T16:21:05.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.730 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-01-25T16:21:05.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-25T16:21:05.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.738 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.740 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-25T16:21:05.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-25T16:21:05.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.753 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-01-25T16:21:05.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-01-25T16:21:05.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-25T16:21:05.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-25T16:21:05.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.779 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-25T16:21:05.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-25T16:21:05.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-25T16:21:05.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:05.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.797 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.799 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-01-25T16:21:05.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:05.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.810 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.811 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.814 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.815 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.819 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:05.820 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.823 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:05.829 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-01-25T16:21:05.830 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.832 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.835 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.838 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.842 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-01-25T16:21:05.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-01-25T16:21:05.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.847 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-01-25T16:21:05.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-25T16:21:05.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-01-25T16:21:05.854 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.856 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.860 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.861 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-01-25T16:21:05.865 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:05.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-01-25T16:21:05.868 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-01-25T16:21:05.869 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-01-25T16:21:05.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-25T16:21:05.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.876 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.880 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-01-25T16:21:05.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-01-25T16:21:05.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:05.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-01-25T16:21:05.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.896 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.899 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.900 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.902 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.903 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.905 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.906 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.908 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.911 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.912 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.914 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.915 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-01-25T16:21:05.916 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-01-25T16:21:05.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.921 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.922 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.925 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.928 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.932 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.935 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.936 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.937 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.942 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-01-25T16:21:05.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.948 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.949 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.956 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.959 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.961 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.962 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.965 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.967 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.968 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.970 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.973 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.975 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.976 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.978 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.979 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.981 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.983 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.984 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.986 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.990 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.991 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.992 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:05.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.994 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.995 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:05.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:05.998 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:05.999 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.003 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.007 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.012 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.015 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.016 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.019 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.023 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:06.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.028 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.032 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.033 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:06.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.043 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.046 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.047 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-01-25T16:21:06.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-01-25T16:21:06.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:06.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-25T16:21:06.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.058 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.064 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.065 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.070 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.074 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.075 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-01-25T16:21:06.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.078 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:06.080 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.083 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.085 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.086 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.093 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.094 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.095 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.096 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.099 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.100 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "drive_group_interval": { 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "drive_group_interval", 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300.0", 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.115 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.116 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.118 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.120 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-01-25T16:21:06.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-01-25T16:21:06.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.131 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.132 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.133 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-01-25T16:21:06.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.137 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-01-25T16:21:06.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.143 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-01-25T16:21:06.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-01-25T16:21:06.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.148 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.149 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.152 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.153 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-01-25T16:21:06.155 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.157 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.161 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.162 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.164 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.170 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.173 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.178 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.182 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-01-25T16:21:06.185 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.187 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-01-25T16:21:06.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-25T16:21:06.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-01-25T16:21:06.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.208 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.212 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-01-25T16:21:06.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.220 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.224 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.226 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.227 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.229 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-01-25T16:21:06.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-25T16:21:06.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.243 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-01-25T16:21:06.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-01-25T16:21:06.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.250 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24955, 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540701272860131327, 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-01-25T16:21:06.262 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.266 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.270 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-01-25T16:21:06.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-01-25T16:21:06.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-01-25T16:21:06.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-01-25T16:21:06.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.286 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-01-25T16:21:06.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-01-25T16:21:06.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-01-25T16:21:06.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-01-25T16:21:06.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-01-25T16:21:06.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:06.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.300 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.301 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.306 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.310 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.320 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-01-25T16:21:06.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-01-25T16:21:06.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-25T16:21:06.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-01-25T16:21:06.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-01-25T16:21:06.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-01-25T16:21:06.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.352 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-01-25T16:21:06.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-01-25T16:21:06.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.357 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.358 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-01-25T16:21:06.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.364 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:06.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-01-25T16:21:06.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-25T16:21:06.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.372 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-01-25T16:21:06.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.375 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-01-25T16:21:06.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.387 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-25T16:21:06.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-25T16:21:06.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:06.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-01-25T16:21:06.399 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-01-25T16:21:06.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-25T16:21:06.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.406 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-01-25T16:21:06.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-01-25T16:21:06.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-01-25T16:21:06.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.419 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-01-25T16:21:06.421 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.423 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.427 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.438 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.445 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.448 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.453 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-25T16:21:06.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.461 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-25T16:21:06.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.476 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-25T16:21:06.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-25T16:21:06.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-25T16:21:06.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-25T16:21:06.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-25T16:21:06.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.495 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-25T16:21:06.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-01-25T16:21:06.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:06.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.513 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.517 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-01-25T16:21:06.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.525 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:06.527 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:06.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-01-25T16:21:06.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.539 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.543 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-01-25T16:21:06.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-01-25T16:21:06.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-01-25T16:21:06.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-25T16:21:06.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.558 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.560 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:06.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.568 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-01-25T16:21:06.572 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-01-25T16:21:06.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-01-25T16:21:06.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.580 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.584 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-01-25T16:21:06.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-01-25T16:21:06.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-01-25T16:21:06.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-01-25T16:21:06.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-01-25T16:21:06.597 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.599 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.603 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.604 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.606 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.608 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.612 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.615 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-01-25T16:21:06.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-25T16:21:06.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.620 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.622 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.624 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.631 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.635 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.637 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-01-25T16:21:06.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-01-25T16:21:06.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.648 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.651 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.655 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.656 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.658 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.662 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.663 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.667 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.671 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.672 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:06.674 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.677 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.681 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-01-25T16:21:06.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-01-25T16:21:06.685 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.686 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.688 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.689 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.692 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.693 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.697 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.701 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.702 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.706 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.710 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.714 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-01-25T16:21:06.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.721 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.724 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.729 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.732 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.736 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.737 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.741 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.742 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.744 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-01-25T16:21:06.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:06.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-25T16:21:06.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:06.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-01-25T16:21:06.755 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:06.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.758 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.760 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.764 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-01-25T16:21:06.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.772 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.776 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.777 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.780 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.781 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:06.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.788 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.790 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.794 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.797 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "drive_group_interval": { 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "drive_group_interval", 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300.0", 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.799 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.800 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.801 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.802 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.804 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-01-25T16:21:06.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.809 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.816 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.819 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.822 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-01-25T16:21:06.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.829 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.830 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-01-25T16:21:06.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-01-25T16:21:06.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.842 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.843 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.846 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.847 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.851 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.855 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.856 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.857 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.858 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.860 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.861 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.862 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.864 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.865 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.867 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-25T16:21:06.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.872 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.875 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.876 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.878 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.879 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-01-25T16:21:06.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-25T16:21:06.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.883 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-01-25T16:21:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-25T16:21:06.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.891 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.892 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.893 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.895 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.896 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.897 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.902 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.903 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.905 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.906 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-01-25T16:21:06.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.911 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.912 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.915 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.918 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.919 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.921 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.924 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.928 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.929 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.930 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-01-25T16:21:06.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.932 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.933 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.934 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-01-25T16:21:06.936 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-01-25T16:21:06.937 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-01-25T16:21:06.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.939 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.945 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.946 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.947 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.948 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-01-25T16:21:06.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.954 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-01-25T16:21:06.955 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "restful", 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "snap_schedule" 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-01-25T16:21:06.956 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.960 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to", 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-01-25T16:21:06.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-01-25T16:21:06.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:06.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.974 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-01-25T16:21:06.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-01-25T16:21:06.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.979 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-01-25T16:21:06.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:06.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-01-25T16:21:06.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-01-25T16:21:06.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.988 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.990 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:06.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:06.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.994 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:06.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:06.998 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:06.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-01-25T16:21:07.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-01-25T16:21:07.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap" 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.004 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.005 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-01-25T16:21:07.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:07.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-01-25T16:21:07.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.012 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.013 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-01-25T16:21:07.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-01-25T16:21:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.020 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.021 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.023 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.024 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.026 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.028 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.029 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-01-25T16:21:07.030 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-01-25T16:21:07.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-01-25T16:21:07.032 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.034 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.035 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-01-25T16:21:07.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-01-25T16:21:07.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-01-25T16:21:07.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-01-25T16:21:07.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.048 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.049 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-01-25T16:21:07.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-01-25T16:21:07.051 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.053 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.054 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-01-25T16:21:07.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-01-25T16:21:07.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-01-25T16:21:07.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.059 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.060 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:07.065 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-01-25T16:21:07.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-01-25T16:21:07.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.068 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.069 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-01-25T16:21:07.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-01-25T16:21:07.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.072 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.075 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.078 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-01-25T16:21:07.079 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-01-25T16:21:07.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.081 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.084 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-01-25T16:21:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-01-25T16:21:07.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-01-25T16:21:07.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.091 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-01-25T16:21:07.094 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-01-25T16:21:07.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.097 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.098 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-01-25T16:21:07.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-01-25T16:21:07.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.102 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-01-25T16:21:07.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-01-25T16:21:07.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.109 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.111 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.114 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.115 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.116 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.117 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.118 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-01-25T16:21:07.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-01-25T16:21:07.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-01-25T16:21:07.121 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.123 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-01-25T16:21:07.128 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-01-25T16:21:07.133 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.140 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-01-25T16:21:07.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.144 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-01-25T16:21:07.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-01-25T16:21:07.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-01-25T16:21:07.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-01-25T16:21:07.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.172 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-01-25T16:21:07.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-01-25T16:21:07.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.180 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-01-25T16:21:07.185 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.188 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-01-25T16:21:07.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.196 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.208 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-01-25T16:21:07.209 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:07.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:07.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.215 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:07.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-01-25T16:21:07.218 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.224 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.226 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.230 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.234 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:07.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-01-25T16:21:07.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.240 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-01-25T16:21:07.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.244 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.246 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.250 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-01-25T16:21:07.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-01-25T16:21:07.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-01-25T16:21:07.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-01-25T16:21:07.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.265 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.266 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.270 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-01-25T16:21:07.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-01-25T16:21:07.274 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.283 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.285 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.286 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.289 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.293 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.298 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.306 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.310 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.311 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-01-25T16:21:07.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.316 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.320 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-01-25T16:21:07.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-01-25T16:21:07.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.325 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.326 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.333 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.337 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.340 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.341 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.345 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.350 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.351 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.354 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.355 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.358 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.359 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.362 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.363 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.367 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.368 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.369 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.371 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.376 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.379 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.387 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.389 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.392 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.393 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.396 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-01-25T16:21:07.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.400 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:07.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.405 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.409 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-01-25T16:21:07.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-01-25T16:21:07.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.419 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.420 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.421 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.423 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.427 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-01-25T16:21:07.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.432 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-01-25T16:21:07.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-01-25T16:21:07.438 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.441 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.449 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-01-25T16:21:07.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-01-25T16:21:07.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-01-25T16:21:07.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.458 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.459 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.461 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.462 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.463 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.466 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.470 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.471 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.475 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.478 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "drive_group_interval": { 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "drive_group_interval", 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300.0", 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.489 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.492 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.493 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.496 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.498 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-01-25T16:21:07.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.510 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.511 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-01-25T16:21:07.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-01-25T16:21:07.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.518 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-01-25T16:21:07.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.523 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.525 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.527 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.530 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.534 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.535 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.536 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.540 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.544 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.556 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.561 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-01-25T16:21:07.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-01-25T16:21:07.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:07.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-01-25T16:21:07.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.582 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.585 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.586 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.587 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.588 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.590 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.591 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.593 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.599 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.604 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.605 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.608 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.616 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-01-25T16:21:07.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-01-25T16:21:07.631 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-01-25T16:21:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.634 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-01-25T16:21:07.635 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-01-25T16:21:07.636 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-01-25T16:21:07.637 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-01-25T16:21:07.638 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-01-25T16:21:07.639 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-01-25T16:21:07.640 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-01-25T16:21:07.641 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 89, 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-01-25T16:21:07.642 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4209976795 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2529319922 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4069389363 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3343574197 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 32416156 2024-01-25T16:21:07.647 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-01-25T16:21:07.648 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4223163698 2024-01-25T16:21:07.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.649 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.649 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:07.649 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-01-25T16:21:07.649 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:21:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 msgr2=0x7f13c40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 0x7f13c40a3940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f13dc1033c0 tx=0x7f13d0005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 msgr2=0x7f13dc101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:07.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00e9f0 tx=0x7f13cc00edf0 comp rx=0 tx=0).stop 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 shutdown_connections 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f13c40a1480 0x7f13c40a3940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f13dc1023b0 0x7f13dc1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13dc104fc0 0x7f13dc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 --2- 172.21.15.33:0/342322535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13dc100940 0x7f13dc101930 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 >> 172.21.15.33:0/342322535 conn(0x7f13dc074f40 msgr2=0x7f13dc103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 shutdown_connections 2024-01-25T16:21:07.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:05.554+0000 7f13e1b25640 1 -- 172.21.15.33:0/342322535 wait complete. 2024-01-25T16:21:07.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:05 smithi033 ceph-mon[31417]: pgmap v422: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-01-25T16:21:07.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/263314667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2541038570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.652 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:05 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1968683744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.653 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:05 smithi076 ceph-mon[35642]: pgmap v422: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-01-25T16:21:07.653 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:05 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/263314667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.653 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:05 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2541038570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.653 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:05 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1968683744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.653 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:06 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/342322535' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:05 smithi152 ceph-mon[36015]: pgmap v422: 97 pgs: 97 active+clean; 126 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.8 MiB/s wr, 337 op/s 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:05 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/263314667' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:05 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2541038570' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:05 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1968683744' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:06 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/342322535' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-25T16:21:07.654 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:07 smithi152 ceph-mon[36015]: pgmap v423: 97 pgs: 97 active+clean; 109 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-01-25T16:21:07.661 DEBUG:tasks.check_counter:Getting stats from a 2024-01-25T16:21:07.661 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.a perf dump 2024-01-25T16:21:07.661 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:06 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/342322535' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-01-25T16:21:07.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:07 smithi076 ceph-mon[35642]: pgmap v423: 97 pgs: 97 active+clean; 109 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-01-25T16:21:07.686 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:07 smithi033 ceph-mon[31417]: pgmap v423: 97 pgs: 97 active+clean; 109 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 2.5 MiB/s wr, 285 op/s 2024-01-25T16:21:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:21:09.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:09 smithi152 ceph-mon[36015]: pgmap v424: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:09.639 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:09 smithi033 ceph-mon[31417]: pgmap v424: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:09.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:09 smithi076 ceph-mon[35642]: pgmap v424: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-01-25T16:21:09.767 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001285676, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000231108, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:09.768 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:09.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 190, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-01-25T16:21:09.770 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 187696, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.012462774, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.008631750, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001733531, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:09.771 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 187, 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.013261022, 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000070914 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:09.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 187696 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 264, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 251, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1573488, 2024-01-25T16:21:09.773 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 150064, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.042746936, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.014553621, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.031493219, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004737619, 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:09.774 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 251, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010407978, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000041466 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.775 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1573488, 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 150064 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-01-25T16:21:09.776 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:09.777 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:09.778 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:09.779 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:09.780 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149057, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 32, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-01-25T16:21:09.781 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:09.782 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:09.783 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 262, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1533780, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 262, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-01-25T16:21:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 262, 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1533780, 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:09.785 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:21:10.479 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-01-25T16:21:10.479 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-01-25T16:21:10.479 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-01-25T16:21:10.480 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-01-25T16:21:10.480 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-01-25T16:21:10.480 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-01-25T16:21:10.480 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-01-25T16:21:10.480 DEBUG:tasks.check_counter:Getting stats from d 2024-01-25T16:21:10.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.d perf dump 2024-01-25T16:21:10.771 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:21:11.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:11 smithi033 ceph-mon[31417]: pgmap v425: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:11.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:11 smithi033 ceph-mon[31417]: osdmap e112: 12 total, 12 up, 12 in 2024-01-25T16:21:11.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:11 smithi152 ceph-mon[36015]: pgmap v425: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:11.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:11 smithi152 ceph-mon[36015]: osdmap e112: 12 total, 12 up, 12 in 2024-01-25T16:21:11.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:11 smithi076 ceph-mon[35642]: pgmap v425: 97 pgs: 97 active+clean; 106 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 2.5 MiB/s wr, 287 op/s 2024-01-25T16:21:11.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:11 smithi076 ceph-mon[35642]: osdmap e112: 12 total, 12 up, 12 in 2024-01-25T16:21:12.489 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:12 smithi033 ceph-mon[31417]: pgmap v427: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.5 MiB/s wr, 85 op/s 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-01-25T16:21:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001168586, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000292437, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 266, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 255, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1572080, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 153360, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.046537071, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.014441485, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.036415733, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004786724, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 254, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011783187, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000046390 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1572080, 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 153360 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 197, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3360, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 189024, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014108780, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009658165, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002070632, 2024-01-25T16:21:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 195, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.011886835, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000060958 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 189024 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149034, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 32, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-01-25T16:21:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 268, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1534560, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 268, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 268, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1534560, 2024-01-25T16:21:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:12.507 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:21:12.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:12 smithi152 ceph-mon[36015]: pgmap v427: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.5 MiB/s wr, 85 op/s 2024-01-25T16:21:12.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:12 smithi076 ceph-mon[35642]: pgmap v427: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.5 MiB/s wr, 85 op/s 2024-01-25T16:21:13.232 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-01-25T16:21:13.233 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-01-25T16:21:13.233 DEBUG:tasks.check_counter:Getting stats from g 2024-01-25T16:21:13.234 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.g perf dump 2024-01-25T16:21:13.481 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:21:14.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:14 smithi152 ceph-mon[36015]: pgmap v428: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 84 op/s 2024-01-25T16:21:14.630 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:14 smithi033 ceph-mon[31417]: pgmap v428: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 84 op/s 2024-01-25T16:21:14.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:14 smithi076 ceph-mon[35642]: pgmap v428: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 3.5 MiB/s wr, 84 op/s 2024-01-25T16:21:15.145 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:21:15.145 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 262, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 251, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1570992, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 150864, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.044605312, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.014477721, 2024-01-25T16:21:15.146 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.032953481, 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004627545, 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 250, 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010395190, 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000041580 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:15.147 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1570992, 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 150864 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:15.148 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 192, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 188048, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.013327469, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.009348892, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001820772, 2024-01-25T16:21:15.149 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 189, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010510133, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000055609 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:15.150 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 188048 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 4, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 4, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3040, 2024-01-25T16:21:15.151 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 448, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.000620524, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000032130, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000568328, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:15.152 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000727880, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000242626 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.153 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:15.154 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:15.155 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:15.156 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:15.157 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:15.158 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149017, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:15.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-01-25T16:21:15.160 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:15.161 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 264, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1534040, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 264, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-01-25T16:21:15.162 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 264, 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1534040, 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:15.163 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:21:15.847 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-01-25T16:21:15.847 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-01-25T16:21:15.847 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-01-25T16:21:15.847 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-01-25T16:21:15.847 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-01-25T16:21:15.848 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-01-25T16:21:15.848 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-01-25T16:21:15.848 DEBUG:tasks.check_counter:Getting stats from j 2024-01-25T16:21:15.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.j perf dump 2024-01-25T16:21:16.091 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.a/config 2024-01-25T16:21:17.406 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:17 smithi033 ceph-mon[31417]: pgmap v429: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.2 MiB/s wr, 69 op/s 2024-01-25T16:21:17.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:17 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:21:17.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:17 smithi152 ceph-mon[36015]: pgmap v429: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.2 MiB/s wr, 69 op/s 2024-01-25T16:21:17.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:17 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:21:17.647 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:17.648 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001320834, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000311451, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:17.649 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 6, 2024-01-25T16:21:17.650 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 199, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 3360, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 189376, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.014961293, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.010544897, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.002202314, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:17.651 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 197, 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.012704381, 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000064489 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:17.652 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 189376 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 268, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 257, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 1572624, 2024-01-25T16:21:17.653 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 154608, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.045586726, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.015043721, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.035005709, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.004661052, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 256, 2024-01-25T16:21:17.654 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.010891346, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000042544 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1572624, 2024-01-25T16:21:17.655 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 154608 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:17.656 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:17.657 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:17.658 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:17.659 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:17.660 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 149017, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-01-25T16:21:17.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:17.662 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-01-25T16:21:17.663 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 270, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 1534820, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 270, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-01-25T16:21:17.664 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 270, 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 1534820, 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:17.665 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-01-25T16:21:17.666 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-01-25T16:21:17.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:17 smithi076 ceph-mon[35642]: pgmap v429: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 3.2 MiB/s wr, 69 op/s 2024-01-25T16:21:17.666 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:17 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-01-25T16:21:18.390 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:18 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:21:18.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:18 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:21:18.391 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:18 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:21:18.417 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-01-25T16:21:18.417 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-01-25T16:21:18.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-01-25T16:21:18.417 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-01-25T16:21:18.418 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-01-25T16:21:18.418 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-01-25T16:21:18.418 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-01-25T16:21:18.418 DEBUG:tasks.check_counter:Getting stats from b 2024-01-25T16:21:18.418 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.b perf dump 2024-01-25T16:21:18.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:18 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:21:18.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:18 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:21:18.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:18 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:21:18.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:18 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-01-25T16:21:18.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:18 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-01-25T16:21:18.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:18 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' 2024-01-25T16:21:18.756 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:21:19.414 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:19 smithi076 ceph-mon[35642]: pgmap v430: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:19.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:19 smithi033 ceph-mon[31417]: pgmap v430: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:19.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:19 smithi152 ceph-mon[36015]: pgmap v430: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:20.503 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:20 smithi076 ceph-mon[35642]: pgmap v431: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:20.503 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-01-25T16:21:20.503 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 209191, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 333048, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 41992553, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 473144279, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 11, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 15.439729582, 2024-01-25T16:21:20.504 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 7.441963310, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 7.199143858, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 1.389079749, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 333035, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 15.009765238, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000045069 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.505 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 451697, 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.061351624, 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000135 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 426368, 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 135520 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:20.506 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 410335, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 543660, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 145981841, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 673463577, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 130, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 22.708488817, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 10.456445415, 2024-01-25T16:21:20.507 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 17.662285194, 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 1.471990101, 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 543660, 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 26.602862989, 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000048932 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:20.508 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 789645, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.086712068, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000109 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 132665376, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 445393648 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 159462, 2024-01-25T16:21:20.509 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 297846, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 28708861, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 242314640, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 13, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 6, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 10.420867574, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 4.762871748, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 5.106521483, 2024-01-25T16:21:20.510 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.644428628, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 297843, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 13.337596180, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000044780 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 411329, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.058774163, 2024-01-25T16:21:20.511 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000142 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 43456, 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 510192 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-01-25T16:21:20.512 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 110406, 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 69.091623223, 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000625795 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.513 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 47272, 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5.620043577, 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000118887 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.514 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 266206, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 262500, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 262500, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 561.624735839, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002139522 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-01-25T16:21:20.515 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 3706, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 2074, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 9, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 10, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 2157, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 774, 2024-01-25T16:21:20.516 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 1355, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 28, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 1035, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 311, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 477, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 477, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 5, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 384293, 2024-01-25T16:21:20.517 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 253862, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 3705, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 108, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 2759, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 3, 2024-01-25T16:21:20.518 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 42, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 1, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 22, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 87, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 17335589, 2024-01-25T16:21:20.519 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 6, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:20.520 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 3, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 45687, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 100668, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 1110, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 89978, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 23918, 2024-01-25T16:21:20.521 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 6861, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 191234 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 27, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 108, 2024-01-25T16:21:20.522 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 204, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 395, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 97, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 29472, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 29077, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:20.523 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 3, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 19, 2024-01-25T16:21:20.524 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 141812, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 140313, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 140313, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 1499, 2024-01-25T16:21:20.525 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 3214, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 238, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 236, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 236, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 2, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 1, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 5, 2024-01-25T16:21:20.526 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 605564057, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 610162967, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 56251, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 138.748593830, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002466597 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.527 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1171, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 30422, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 29251, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 51, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 120, 2024-01-25T16:21:20.528 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 69, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 2157, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 63564, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 61407, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 380, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 32685, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 32305, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 82000, 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 207228 2024-01-25T16:21:20.529 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 424693, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 6426, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 266206, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 3710, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 12528, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 29559, 2024-01-25T16:21:20.530 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 38.040667016, 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001286940 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 10711, 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 50.845107638, 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.004746999 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.531 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.532 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 65890, 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 231.288792422, 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003510226 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:20.533 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.534 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.535 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.536 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:20.536 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.536 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 14, 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.005755323, 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000411094 2024-01-25T16:21:20.537 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 20, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.023022701, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001151135 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.538 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 14, 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.127030056, 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.009073575 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.539 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 120247, 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 184.926800669, 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001537891 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.540 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5462, 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 16.323047054, 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002988474 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.541 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.542 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8, 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.222227471, 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.027778433 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:20.543 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7, 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.021290914, 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.003041559 2024-01-25T16:21:20.544 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1088, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.833324156, 2024-01-25T16:21:20.545 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000765922 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 8, 2024-01-25T16:21:20.546 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.021305942, 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002663242 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.547 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 29472, 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 38.946364477, 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001321470 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:20.548 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 6, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 4, 2024-01-25T16:21:20.549 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 45514, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 22757, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 805, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 360, 2024-01-25T16:21:20.550 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 360, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:20.551 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:20.552 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:20.553 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:20.554 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 1262310, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 1202, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:20.555 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 190, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 4940605, 2024-01-25T16:21:20.556 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 103351, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-01-25T16:21:20.557 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 105384, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 632605681, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 37, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 105384, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 105384, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 904.224344563, 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.008580281 2024-01-25T16:21:20.558 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 105384, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 138719 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 105384, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 1036, 2024-01-25T16:21:20.559 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 104348, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 1962, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 14013, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 1036, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 57287, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 752, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-01-25T16:21:20.560 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 9, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 29935, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-01-25T16:21:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 28026, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-01-25T16:21:20.562 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-01-25T16:21:20.563 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-01-25T16:21:20.564 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 112, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 27, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 4365, 2024-01-25T16:21:20.565 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 1334 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 3, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 3, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.566 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 10, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 10, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 29077, 2024-01-25T16:21:20.567 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 29077, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 778984, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 129828573, 2024-01-25T16:21:20.568 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 778984, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 778984, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 129828573, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.569 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:20.570 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 105384, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 635547578, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 105384, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 635547578, 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:20.571 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:20.572 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 105384, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 105384, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 105384, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 105384, 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:20.573 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:20.574 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 29077, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 2941897, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 29077, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 1036, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 2941897, 2024-01-25T16:21:20.575 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x560958cc06e0": { 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:20.576 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 141812, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 605968663, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 141812, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 56251, 2024-01-25T16:21:20.577 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 605968663, 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:20.578 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-01-25T16:21:20.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:20 smithi152 ceph-mon[36015]: pgmap v431: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:20.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:20 smithi033 ceph-mon[31417]: pgmap v431: 97 pgs: 97 active+clean; 123 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 3.2 MiB/s wr, 68 op/s 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds.imported=1 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=27 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=14 2024-01-25T16:21:21.258 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=8 2024-01-25T16:21:21.258 DEBUG:tasks.check_counter:Getting stats from e 2024-01-25T16:21:21.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.e perf dump 2024-01-25T16:21:21.506 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:21:22.600 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:22 smithi076 ceph-mon[35642]: pgmap v432: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.4 MiB/s wr, 83 op/s 2024-01-25T16:21:22.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:22 smithi152 ceph-mon[36015]: pgmap v432: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.4 MiB/s wr, 83 op/s 2024-01-25T16:21:22.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:22 smithi033 ceph-mon[31417]: pgmap v432: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 3.4 MiB/s wr, 83 op/s 2024-01-25T16:21:23.621 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-01-25T16:21:23.621 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:23.621 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 814, 2024-01-25T16:21:23.621 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 816, 2024-01-25T16:21:23.621 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 14374250, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 253218, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.072605378, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.022984905, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.024457702, 2024-01-25T16:21:23.622 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.018444113, 2024-01-25T16:21:23.623 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:23.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 814, 2024-01-25T16:21:23.623 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.028253325, 2024-01-25T16:21:23.623 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000034709 2024-01-25T16:21:23.623 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:23.624 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 1413, 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 1377, 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 31329585, 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 498526, 2024-01-25T16:21:23.625 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.139436413, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.033395414, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.079384972, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.031064277, 2024-01-25T16:21:23.626 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1377, 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.038548596, 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000027994 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:23.627 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1695360, 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 160464 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.628 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 1350, 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 1545, 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 13002143, 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 923964, 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 7, 2024-01-25T16:21:23.629 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:23.631 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.126411537, 2024-01-25T16:21:23.631 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.038492145, 2024-01-25T16:21:23.631 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.037577549, 2024-01-25T16:21:23.631 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.038255027, 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1542, 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.062758688, 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000040699 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:23.632 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 504736 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-01-25T16:21:23.633 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:23.634 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5593, 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.122386099, 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000021882 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:23.635 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.636 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 0, 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 0, 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.637 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-01-25T16:21:23.638 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 4463, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 3, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 4460, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 0, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 4459, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 0, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 0, 2024-01-25T16:21:23.639 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 0, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 4, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 0, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 0, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 0, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 0, 2024-01-25T16:21:23.640 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 0, 2024-01-25T16:21:23.641 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 0, 2024-01-25T16:21:23.642 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:23.643 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:23.644 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 0, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 0, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 0, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 0 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.645 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 0, 2024-01-25T16:21:23.646 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:23.647 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-01-25T16:21:23.648 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 27644, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-01-25T16:21:23.649 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 0, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 35, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 4, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 60302183, 2024-01-25T16:21:23.650 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 60294419, 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 27644 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-01-25T16:21:23.651 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 241, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 4478, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 4237, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 23, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 24, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 1, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 4463, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 4485, 2024-01-25T16:21:23.652 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 22, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 0, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 0, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 0, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 56084, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 207228 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 0, 2024-01-25T16:21:23.653 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 0, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 0, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 0, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.654 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.655 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.656 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-01-25T16:21:23.657 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.658 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.659 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-01-25T16:21:23.660 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.661 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.661 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.662 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.663 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-01-25T16:21:23.664 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.665 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.666 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:23.667 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.668 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.669 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.670 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.671 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.672 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.673 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.674 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-01-25T16:21:23.675 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 5, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 3, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 5, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 0, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 0, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 4018, 2024-01-25T16:21:23.676 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:23.677 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:23.678 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:23.679 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:23.680 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:23.681 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:23.682 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 224394, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 296, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:23.683 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 190, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:23.684 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 3789832, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 34123, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-01-25T16:21:23.685 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-01-25T16:21:23.686 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 3249, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 21, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 3, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 3249, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3249, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.808944322, 2024-01-25T16:21:23.687 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000556769 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3249, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3250 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 3249, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 3249, 2024-01-25T16:21:23.688 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 0, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 1749, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 0, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 1498, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 0, 2024-01-25T16:21:23.689 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:23.690 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 1, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-01-25T16:21:23.691 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-01-25T16:21:23.692 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-01-25T16:21:23.693 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 112, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 26, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 8, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 9, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 1, 2024-01-25T16:21:23.694 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 0, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 2, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 0 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 0, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-01-25T16:21:23.695 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 0, 2024-01-25T16:21:23.696 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 0, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 0, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 0, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:23.697 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 3573, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 58365126, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 3573, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3573, 2024-01-25T16:21:23.698 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 58365126, 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.699 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:23.700 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 3249, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 56110288, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3249, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 56110288, 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:23.701 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:23.702 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 3249, 2024-01-25T16:21:23.703 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 3249, 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 3249, 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 3249, 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.704 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:23.705 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-01-25T16:21:23.706 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.707 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55efe0172960": { 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:23.709 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:23.710 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:23.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.710 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:23.710 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-01-25T16:21:24.455 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-01-25T16:21:24.455 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-01-25T16:21:24.455 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-01-25T16:21:24.455 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-01-25T16:21:24.455 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-01-25T16:21:24.456 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-01-25T16:21:24.456 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-01-25T16:21:24.456 DEBUG:tasks.check_counter:Getting stats from h 2024-01-25T16:21:24.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.h perf dump 2024-01-25T16:21:24.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:24 smithi152 ceph-mon[36015]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-01-25T16:21:24.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:24 smithi076 ceph-mon[35642]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-01-25T16:21:24.692 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:21:24.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:24 smithi033 ceph-mon[31417]: pgmap v433: 97 pgs: 97 active+clean; 122 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-01-25T16:21:26.410 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 292400, 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 167966, 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 141155980, 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 58245071, 2024-01-25T16:21:26.411 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 10, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 4, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 11.831355072, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 5.249747620, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 8.842128225, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.112626452, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 167966, 2024-01-25T16:21:26.412 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 2.139699974, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000012738 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 450843, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.049396412, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000109 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 160, 2024-01-25T16:21:26.413 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 377936 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 5508, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 5502, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 2992600, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 27185907, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 8, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-01-25T16:21:26.414 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.399804148, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.131527559, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.118908525, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.136551340, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 5496, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.138288064, 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000025161 2024-01-25T16:21:26.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 48, 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000009348, 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000194 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1700992, 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 171856 2024-01-25T16:21:26.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 62744, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 89096, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 21708784, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 54450848, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 17, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 7, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 3.798990324, 2024-01-25T16:21:26.417 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 1.721194577, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 2.862567262, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.088123843, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 89092, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 1.560443105, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000017514 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.418 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 134512, 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.018079008, 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000134 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 20306784, 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 48534080 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-01-25T16:21:26.419 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 1, 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-MDSRank": { 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 14477, 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 9.611571146, 2024-01-25T16:21:26.420 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000663920 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "queue_len": 0, 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "complete_latency": { 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7971, 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.447472971, 2024-01-25T16:21:26.421 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000056137 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": { 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "request": 43903, 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "reply": 26093, 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "reply_latency": { 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 26093, 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 26.174855301, 2024-01-25T16:21:26.422 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001003137 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_reply": 0, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "forward": 17810, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_commit": 11, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_split": 0, 2024-01-25T16:21:26.423 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_merge": 0, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes": 257, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_top": 8, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_bottom": 239, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pin_tail": 10, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_pinned": 245, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_expired": 26, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "inodes_with_caps": 59, 2024-01-25T16:21:26.424 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": 59, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "subtrees": 5, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse": 81141, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_hit": 31211, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_forward": 17809, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_discover": 17, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:26.425 INFO:teuthology.orchestra.run.smithi076.stdout: "traverse_lock": 1614, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "load_cent": 0, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "q": 0, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "exported": 1, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "exported_inodes": 22, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "imported": 3, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "imported_inodes": 42, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:26.426 INFO:teuthology.orchestra.run.smithi076.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rfiles": 87, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rbytes": 17335589, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "root_rsnaps": 6, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:26.427 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_inode_file_caps": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_revoke": 4120, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_grant": 21728, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:26.428 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flushsnap_ack": 347, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps": 11541, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_caps_dirty": 4360, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_cap_release": 1577, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "process_request_cap_release": 23981 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_cache": { 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update": 144, 2024-01-25T16:21:26.429 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_update_receipt": 42, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_try_discover": 28, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_send_discover": 32, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "dir_handle_discover": 253, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays": 44, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_delayed": 0, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_created": 4314, 2024-01-25T16:21:26.430 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_enqueued": 4270, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "strays_migrated": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_started": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "recovery_completed": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:26.431 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_exportdir": 1, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_flush": 0, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragmentdir": 0, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ireq_inodestats": 0 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_log": { 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "evlrg": 0, 2024-01-25T16:21:26.432 INFO:teuthology.orchestra.run.smithi076.stdout: "evadd": 27734, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "evex": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "evtrm": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "ev": 27734, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "evexg": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "evexd": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "segadd": 35, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "segex": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "segtrm": 0, 2024-01-25T16:21:26.433 INFO:teuthology.orchestra.run.smithi076.stdout: "seg": 35, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "segmjr": 4, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "segexg": 0, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "segexd": 0, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "expos": 4194304, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "wrpos": 60523349, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "rdpos": 0, 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "jlat": { 2024-01-25T16:21:26.434 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 7165, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 13.588570893, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001896520 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "replayed": 0 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_mem": { 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 225, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "ino+": 4489, 2024-01-25T16:21:26.435 INFO:teuthology.orchestra.run.smithi076.stdout: "ino-": 4264, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dir": 46, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dir+": 53, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dir-": 7, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dn": 232, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dn+": 7726, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "dn-": 7494, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "cap": 59, 2024-01-25T16:21:26.436 INFO:teuthology.orchestra.run.smithi076.stdout: "cap+": 4334, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "cap-": 4275, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "rss": 67532, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "heap": 199036 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_server": { 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_client_request": 66764, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "dispatch_server_request": 0, 2024-01-25T16:21:26.437 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_request": 43903, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_client_session": 1109, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "handle_peer_request": 6426, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "req_create_latency": { 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4314, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 8.342538951, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001933829 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getattr_latency": { 2024-01-25T16:21:26.438 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000204299, 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000204299 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.439 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "req_link_latency": { 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookup_latency": { 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 9647, 2024-01-25T16:21:26.440 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5.690192795, 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000589840 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupino_latency": { 2024-01-25T16:21:26.441 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000160230, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000160230 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupname_latency": { 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.442 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.443 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.444 INFO:teuthology.orchestra.run.smithi076.stdout: "req_lssnap_latency": { 2024-01-25T16:21:26.444 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mkdir_latency": { 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.445 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mknod_latency": { 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "req_mksnap_latency": { 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.446 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "req_open_latency": { 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "req_readdir_latency": { 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 6199, 2024-01-25T16:21:26.447 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.098080073, 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000499770 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rename_latency": { 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 1614, 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 3.265784075, 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002023410 2024-01-25T16:21:26.448 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.449 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmdir_latency": { 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:26.450 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.451 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setattr_latency": { 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 3, 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.007492579, 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002497526 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:26.452 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.453 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setlayout_latency": { 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "req_setxattr_latency": { 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.454 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "req_symlink_latency": { 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "req_unlink_latency": { 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 4314, 2024-01-25T16:21:26.455 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 5.770402299, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.001337599 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.456 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_sessions": { 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "session_count": 2, 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "session_add": 5, 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "session_remove": 3, 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_open": 2, 2024-01-25T16:21:26.457 INFO:teuthology.orchestra.run.smithi076.stdout: "sessions_stale": 0, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "total_load": 4061, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "average_load": 2030, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "avg_session_uptime": 814, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:26.458 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:26.459 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:26.460 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:26.461 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 1122021, 2024-01-25T16:21:26.462 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 518, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 264, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 3, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:26.463 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 190, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 1006196, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 22925, 2024-01-25T16:21:26.464 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "objecter": { 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "op_active": 0, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "op_laggy": 0, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send": 13031, 2024-01-25T16:21:26.465 INFO:teuthology.orchestra.run.smithi076.stdout: "op_send_bytes": 57118140, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "op_resend": 27, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "op_reply": 13031, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "op_latency": { 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 13031, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 30.038080280, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.002305124 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "op_inflight": 0, 2024-01-25T16:21:26.466 INFO:teuthology.orchestra.run.smithi076.stdout: "oplen_avg": { 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 13031, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 13343 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "op": 13031, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "op_r": 533, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "op_w": 12498, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "op_rmw": 0, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "op_pg": 0, 2024-01-25T16:21:26.467 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_stat": 0, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_create": 10, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_read": 533, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_write": 7698, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writefull": 333, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_writesame": 0, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_append": 0, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_zero": 0, 2024-01-25T16:21:26.468 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_truncate": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_delete": 4295, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_mapext": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_getxattr": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_setxattr": 20, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:26.469 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_call": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_watch": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_notify": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "osdop_other": 0, 2024-01-25T16:21:26.470 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_active": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_send": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_resend": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "linger_ping": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_active": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_send": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolop_resend": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_active": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_send": 0, 2024-01-25T16:21:26.471 INFO:teuthology.orchestra.run.smithi076.stdout: "poolstat_resend": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_active": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_send": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "statfs_resend": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "command_active": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "command_send": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "command_resend": 0, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "map_epoch": 112, 2024-01-25T16:21:26.472 INFO:teuthology.orchestra.run.smithi076.stdout: "map_full": 0, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "map_inc": 26, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_sessions": 12, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_open": 12, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_session_close": 0, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_laggy": 0, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_wr": 308, 2024-01-25T16:21:26.473 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_rd": 0, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_del": 146 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "oft": { 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_objs": 1, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_kv_pairs": 0, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_updates": 0, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "omap_total_removes": 0 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.474 INFO:teuthology.orchestra.run.smithi076.stdout: "osd-slow-ops": { 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "slow_ops_count": 0 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "purge_queue": { 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_ops_high_water": 10, 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing": 0, 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executing_high_water": 10, 2024-01-25T16:21:26.475 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed_ops": 4270, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_executed": 4270, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 360648, 2024-01-25T16:21:26.476 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 130574790, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 360648, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 360648, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 130574790, 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:26.477 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:26.478 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 13031, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 57550186, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 13031, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 57550186, 2024-01-25T16:21:26.479 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:26.480 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 1024, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 0, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 0, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 13031, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 13031, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 13031, 2024-01-25T16:21:26.481 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 13031, 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.482 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 4270, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 432046, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 4270, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:26.483 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 533, 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 432046, 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.484 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-write_buf_throttle-0x55d35ed366e0": { 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 22516, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 3758096384, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 27734, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 56329045, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:26.485 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 27734, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 7165, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 56306529, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:26.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:26.487 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-01-25T16:21:27.099 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-01-25T16:21:27.099 INFO:tasks.check_counter:Daemon mds.h mds.imported=3 2024-01-25T16:21:27.100 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=144 2024-01-25T16:21:27.100 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=42 2024-01-25T16:21:27.100 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-01-25T16:21:27.100 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-01-25T16:21:27.100 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-01-25T16:21:27.100 DEBUG:tasks.check_counter:Getting stats from k 2024-01-25T16:21:27.100 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.k perf dump 2024-01-25T16:21:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.b/config 2024-01-25T16:21:27.413 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:27 smithi076 ceph-mon[35642]: pgmap v434: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-01-25T16:21:27.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:27 smithi033 ceph-mon[31417]: pgmap v434: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-01-25T16:21:27.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:27 smithi152 ceph-mon[36015]: pgmap v434: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 2.0 MiB/s wr, 69 op/s 2024-01-25T16:21:29.010 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 0, 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 2, 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 0, 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 0, 2024-01-25T16:21:29.011 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.001096122, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.000000000, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.000254820, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:29.012 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:29.013 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 271, 2024-01-25T16:21:29.014 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 260, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 1573440, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 156480, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.044456616, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.012875065, 2024-01-25T16:21:29.015 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.035291108, 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.005809567, 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 259, 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.010131577, 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000039118 2024-01-25T16:21:29.016 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 1573440, 2024-01-25T16:21:29.017 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 156480 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_messages": 6, 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages": 201, 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_bytes": 3360, 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_bytes": 189728, 2024-01-25T16:21:29.018 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_total_time": 0.010537398, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_send_time": 0.006760677, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_recv_time": 0.001838204, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:29.019 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 199, 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.011502346, 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000057800 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:29.020 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: "msgr_send_encrypted_bytes": 189728 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.021 INFO:teuthology.orchestra.run.smithi076.stdout: "cct": { 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "total_workers": 0, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "unhealthy_workers": 0 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "mempool": { 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:29.022 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:29.023 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:29.024 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:29.025 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_items": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_bytes": 148994, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_anon_items": 31, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:29.026 INFO:teuthology.orchestra.run.smithi076.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_bytes": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_items": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_items": 0, 2024-01-25T16:21:29.027 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "pgmap_items": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_co_items": 2, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_1_items": 0, 2024-01-25T16:21:29.028 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "unittest_2_items": 0 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "val": 0, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "max": 104857600, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "get_started": 0, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "get": 273, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "get_sum": 1535210, 2024-01-25T16:21:29.029 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "get_or_fail_success": 273, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "take": 0, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "take_sum": 0, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "put": 273, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "put_sum": 1535210, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "wait": { 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "avgcount": 0, 2024-01-25T16:21:29.030 INFO:teuthology.orchestra.run.smithi076.stdout: "sum": 0.000000000, 2024-01-25T16:21:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: "avgtime": 0.000000000 2024-01-25T16:21:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:29.031 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-01-25T16:21:29.031 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-01-25T16:21:29.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:29 smithi033 ceph-mon[31417]: pgmap v435: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-01-25T16:21:29.573 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:29 smithi076 ceph-mon[35642]: pgmap v435: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-01-25T16:21:29.600 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-01-25T16:21:29.601 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-01-25T16:21:29.601 DEBUG:tasks.check_counter:Getting stats from c 2024-01-25T16:21:29.602 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.c perf dump 2024-01-25T16:21:29.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:29 smithi152 ceph-mon[36015]: pgmap v435: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.5 MiB/s wr, 47 op/s 2024-01-25T16:21:29.853 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:21:31.543 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:31 smithi152 ceph-mon[36015]: pgmap v436: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-01-25T16:21:31.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:31 smithi033 ceph-mon[31417]: pgmap v436: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-01-25T16:21:31.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:31 smithi076 ceph-mon[35642]: pgmap v436: 97 pgs: 97 active+clean; 128 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.5 MiB/s wr, 43 op/s 2024-01-25T16:21:31.718 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-01-25T16:21:31.718 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:31.718 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 144, 2024-01-25T16:21:31.718 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 143, 2024-01-25T16:21:31.718 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 12780034, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 47762, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 2, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.020328726, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.003533534, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.011891866, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.003628574, 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:31.719 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 142, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.007221394, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000050854 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.720 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 159296, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 12304 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 78, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 90, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 10800420, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 131201, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 6, 2024-01-25T16:21:31.721 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.013142483, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.002022407, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.008708555, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001714360, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 89, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.005196667, 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000058389 2024-01-25T16:21:31.722 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 108432 2024-01-25T16:21:31.723 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 65, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 67, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 12023789, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 20222, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.009592686, 2024-01-25T16:21:31.724 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.001692352, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.006022091, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.001402152, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 65, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.003034697, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000046687 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:31.725 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-01-25T16:21:31.726 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 400, 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.016225782, 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000040564 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.727 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.728 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.729 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 2427, 2024-01-25T16:21:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 1157, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 1270, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 1046, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 4, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-01-25T16:21:31.731 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-01-25T16:21:31.732 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 87, 2024-01-25T16:21:31.733 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 21253413, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 6, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:31.734 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-01-25T16:21:31.735 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-01-25T16:21:31.736 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-01-25T16:21:31.737 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-01-25T16:21:31.738 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 8291, 2024-01-25T16:21:31.739 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 3, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-01-25T16:21:31.740 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 605564057, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 610561528, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 610559913, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.741 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 8291 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 461, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 2334, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 1873, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 44, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 44, 2024-01-25T16:21:31.742 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 2427, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 2427, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 52268, 2024-01-25T16:21:31.743 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207204 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-01-25T16:21:31.744 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.745 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.746 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.747 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.748 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.749 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.750 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-01-25T16:21:31.751 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.752 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.753 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.754 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-01-25T16:21:31.755 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.756 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.757 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.758 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.759 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.760 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:31.761 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 2, 2024-01-25T16:21:31.762 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 0, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 0, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 2105, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-01-25T16:21:31.763 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:31.764 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:31.765 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:31.766 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:31.767 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 233838, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 602, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-01-25T16:21:31.768 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 190, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:31.769 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 2961141, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 54631, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.770 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 252, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 26, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 4, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 252, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 252, 2024-01-25T16:21:31.771 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.211997502, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000841259 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 252, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 253 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 252, 2024-01-25T16:21:31.772 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 252, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 140, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 109, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-01-25T16:21:31.773 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:31.774 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 2, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:31.775 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-01-25T16:21:31.776 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-01-25T16:21:31.777 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 112, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 2, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 8, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 8, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 0, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-01-25T16:21:31.778 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-01-25T16:21:31.779 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-01-25T16:21:31.780 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 283, 2024-01-25T16:21:31.781 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 35574230, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 283, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 283, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 35574230, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.782 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:31.783 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 252, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 35365629, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 252, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 35365629, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.784 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:31.785 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 252, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 252, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 252, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 252, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:31.787 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:31.788 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x559e74b32960": { 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:31.789 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:31.790 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:31.791 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:31.791 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:31.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.791 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:31.791 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-01-25T16:21:32.491 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-01-25T16:21:32.491 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-01-25T16:21:32.492 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-01-25T16:21:32.492 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-01-25T16:21:32.492 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-01-25T16:21:32.492 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-01-25T16:21:32.492 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-01-25T16:21:32.492 DEBUG:tasks.check_counter:Getting stats from f 2024-01-25T16:21:32.492 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.f perf dump 2024-01-25T16:21:32.798 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:21:33.378 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:33 smithi152 ceph-mon[36015]: pgmap v437: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.2 MiB/s wr, 72 op/s 2024-01-25T16:21:33.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:33 smithi033 ceph-mon[31417]: pgmap v437: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.2 MiB/s wr, 72 op/s 2024-01-25T16:21:33.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:33 smithi076 ceph-mon[35642]: pgmap v437: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.2 MiB/s wr, 72 op/s 2024-01-25T16:21:34.511 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-01-25T16:21:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1272, 2024-01-25T16:21:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1236, 2024-01-25T16:21:34.511 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 2013088, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 455296, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.146969791, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.044436436, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.064831891, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.037814504, 2024-01-25T16:21:34.512 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1236, 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.057091417, 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000046190 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.513 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1695904, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 161712 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 4, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 200, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 876, 2024-01-25T16:21:34.514 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 508796, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 7, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.011527371, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.006636960, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.002728039, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000180454, 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:34.515 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 197, 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.014792399, 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000075088 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.516 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 508112 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 1812, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 1814, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 464380, 2024-01-25T16:21:34.517 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 563539, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.110648514, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.035446282, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.027192121, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.040350779, 2024-01-25T16:21:34.518 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1812, 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.055139231, 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000030430 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.519 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 0, 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 0 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-01-25T16:21:34.520 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4649, 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.113800764, 2024-01-25T16:21:34.521 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000024478 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.522 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 0, 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 0, 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-01-25T16:21:34.523 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 0, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:34.524 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-01-25T16:21:34.525 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 1, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:34.526 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:34.527 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:34.528 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:34.529 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 0 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-01-25T16:21:34.530 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-01-25T16:21:34.531 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-01-25T16:21:34.532 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-01-25T16:21:34.533 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 10, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-01-25T16:21:34.534 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4197183, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 4197042, 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-01-25T16:21:34.535 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 10 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-01-25T16:21:34.536 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 3, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 3, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 1, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 1, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 0, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 0, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 0, 2024-01-25T16:21:34.537 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 0, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 0, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 0, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 48248, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 207228 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 0, 2024-01-25T16:21:34.538 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 0, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 0, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.539 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:34.540 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.541 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.542 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.543 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.545 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-01-25T16:21:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.546 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-01-25T16:21:34.547 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.548 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.549 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:34.550 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.551 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.552 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.553 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.554 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.555 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.556 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.557 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 5, 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 3, 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 4, 2024-01-25T16:21:34.558 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 0, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 0, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 3391, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:34.559 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:34.560 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:34.561 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:34.562 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 181343, 2024-01-25T16:21:34.563 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 47, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:34.564 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 190, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 8320, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 13, 2024-01-25T16:21:34.565 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-01-25T16:21:34.566 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 2758, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 21, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 3, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 2758, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2758, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 1.344983927, 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000487666 2024-01-25T16:21:34.567 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2758, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2759 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 2758, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 2758, 2024-01-25T16:21:34.568 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 1800, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 956, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 0, 2024-01-25T16:21:34.569 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:34.570 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 1, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-01-25T16:21:34.571 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-01-25T16:21:34.572 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-01-25T16:21:34.573 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 112, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 26, 2024-01-25T16:21:34.574 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 4, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 5, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 1, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 0, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 2, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 0 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.575 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 0, 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-01-25T16:21:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-01-25T16:21:34.577 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 3084, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 2181398, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:34.578 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 3084, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 3084, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2181398, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.579 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:34.580 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2758, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2884, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2758, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2884, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.581 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:34.582 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 2758, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 2758, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 2758, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2758, 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:34.583 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:34.584 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:34.585 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x562c7c1be960": { 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:34.586 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:34.587 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:34.588 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:34.588 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:34.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.588 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:34.588 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-01-25T16:21:34.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:34 smithi152 ceph-mon[36015]: pgmap v438: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-01-25T16:21:34.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:34 smithi076 ceph-mon[35642]: pgmap v438: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-01-25T16:21:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:34 smithi033 ceph-mon[31417]: pgmap v438: 97 pgs: 97 active+clean; 127 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 1.5 MiB/s wr, 46 op/s 2024-01-25T16:21:35.260 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-01-25T16:21:35.261 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-01-25T16:21:35.261 DEBUG:tasks.check_counter:Getting stats from i 2024-01-25T16:21:35.262 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.i perf dump 2024-01-25T16:21:35.504 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:21:37.186 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-01-25T16:21:37.186 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:37.186 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 295237, 2024-01-25T16:21:37.186 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 156934, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 144624246, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 25796629, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 9, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 4, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 12.439058619, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 4.827079681, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 10.800939684, 2024-01-25T16:21:37.187 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.010900909, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 156933, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 2.053904818, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000013087 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 451189, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.060981244, 2024-01-25T16:21:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000135 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 7503328, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 2839456 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 43, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 123, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 10682, 2024-01-25T16:21:37.189 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 171818, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 17, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.011146784, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.002935003, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.006988041, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000298903, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 115, 2024-01-25T16:21:37.190 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012839471, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000111647 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 103, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000027628, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000268 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 7600, 2024-01-25T16:21:37.191 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 147280 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 329, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 498, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 162510, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 514882, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 10, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 5, 2024-01-25T16:21:37.192 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.031154851, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.013809891, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.019302589, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000610617, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 497, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.033089960, 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000066579 2024-01-25T16:21:37.193 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 596, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000116536, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000195 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3200, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 381936 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.194 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 1, 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-MDSRank": { 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 65, 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.002939282, 2024-01-25T16:21:37.195 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000045219 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "finisher-PurgeQueue": { 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "queue_len": 0, 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "complete_latency": { 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 2, 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000019961, 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000009980 2024-01-25T16:21:37.196 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "mds": { 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "request": 18486, 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "reply": 5, 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "reply_latency": { 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 5, 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001172794, 2024-01-25T16:21:37.197 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000234558 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_reply": 0, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "forward": 18481, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_complete": 0, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_fetch_keys": 0, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_commit": 11, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_split": 0, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_merge": 0, 2024-01-25T16:21:37.198 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes": 29, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_top": 7, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_bottom": 22, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pin_tail": 0, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_pinned": 19, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_expired": 0, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "inodes_with_caps": 2, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "caps": 2, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "subtrees": 2, 2024-01-25T16:21:37.199 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse": 18658, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_hit": 165, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_forward": 18476, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_discover": 14, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_dir_fetch": 0, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_remote_ino": 0, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "traverse_lock": 0, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "load_cent": 0, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "q": 0, 2024-01-25T16:21:37.200 INFO:teuthology.orchestra.run.smithi152.stdout: "exported": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "exported_inodes": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "imported": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "imported_inodes": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_dir_fetch": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_backtrace_fetch": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "openino_peer_discover": 0, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rfiles": 87, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rbytes": 21253413, 2024-01-25T16:21:37.201 INFO:teuthology.orchestra.run.smithi152.stdout: "root_rsnaps": 6, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_fetch": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_set_tag": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_backtrace_repaired": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_inotable_repaired": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_inodes": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dir_base_inodes": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_dirfrag_rstats": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "scrub_file_inodes": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_inode_file_caps": 0, 2024-01-25T16:21:37.202 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_revoke": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_grant": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_trunc": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "ceph_cap_op_flush_ack": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_caps_dirty": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_cap_release": 0, 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: "process_request_cap_release": 2 2024-01-25T16:21:37.203 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_cache": { 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update": 0, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_update_receipt": 326, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_try_discover": 169, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_send_discover": 317, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "dir_handle_discover": 0, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays": 0, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_delayed": 0, 2024-01-25T16:21:37.204 INFO:teuthology.orchestra.run.smithi152.stdout: "num_strays_enqueuing": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_created": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_enqueued": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_reintegrated": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "strays_migrated": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_processing": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_enqueued": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "num_recovering_prioritized": 0, 2024-01-25T16:21:37.205 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_started": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "recovery_completed": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_enqueue_scrub": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_exportdir": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_flush": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragmentdir": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_fragstats": 0, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "ireq_inodestats": 0 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.206 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_log": { 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evlrg": 0, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evadd": 10, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evex": 0, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evtrm": 0, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "ev": 10, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evexg": 0, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "evexd": 0, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "segadd": 2, 2024-01-25T16:21:37.207 INFO:teuthology.orchestra.run.smithi152.stdout: "segex": 0, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "segtrm": 0, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "seg": 2, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "segmjr": 2, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "segexg": 0, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "segexd": 0, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "expos": 4194304, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "wrpos": 4197183, 2024-01-25T16:21:37.208 INFO:teuthology.orchestra.run.smithi152.stdout: "rdpos": 0, 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "jlat": { 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 9, 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.026550883, 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.002950098 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "replayed": 0 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_mem": { 2024-01-25T16:21:37.209 INFO:teuthology.orchestra.run.smithi152.stdout: "ino": 32, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "ino+": 32, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "ino-": 0, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dir": 43, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dir+": 194, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dir-": 151, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dn": 29, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dn+": 29, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "dn-": 0, 2024-01-25T16:21:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: "cap": 2, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "cap+": 3, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "cap-": 1, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "rss": 53220, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "heap": 199036 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_server": { 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_client_request": 18488, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "dispatch_server_request": 0, 2024-01-25T16:21:37.211 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_request": 18486, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_client_session": 32, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "handle_peer_request": 0, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "req_create_latency": { 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.212 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getattr_latency": { 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 4, 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.001085414, 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000271353 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getfilelock_latency": { 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.213 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "req_link_latency": { 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookup_latency": { 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.214 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookuphash_latency": { 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupino_latency": { 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 1, 2024-01-25T16:21:37.215 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000087380, 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000087380 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupname_latency": { 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupparent_latency": { 2024-01-25T16:21:37.216 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lookupsnap_latency": { 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.217 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "req_lssnap_latency": { 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mkdir_latency": { 2024-01-25T16:21:37.218 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.219 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.219 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.219 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.219 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mknod_latency": { 2024-01-25T16:21:37.219 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "req_mksnap_latency": { 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.220 INFO:teuthology.orchestra.run.smithi152.stdout: "req_open_latency": { 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "req_readdir_latency": { 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.221 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rename_latency": { 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "req_renamesnap_latency": { 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.222 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "req_snapdiff_latency": { 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmdir_latency": { 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.223 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmsnap_latency": { 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "req_rmxattr_latency": { 2024-01-25T16:21:37.224 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setattr_latency": { 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.225 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setdirlayout_latency": { 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setfilelock_latency": { 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.226 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setlayout_latency": { 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "req_setxattr_latency": { 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.227 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "req_symlink_latency": { 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "req_unlink_latency": { 2024-01-25T16:21:37.228 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_revoke_eviction": 0, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "cap_acquisition_throttle": 0, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "req_getvxattr_latency": { 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.229 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_sessions": { 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "session_count": 2, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "session_add": 5, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "session_remove": 3, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_open": 2, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "sessions_stale": 0, 2024-01-25T16:21:37.230 INFO:teuthology.orchestra.run.smithi152.stdout: "total_load": 4, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "average_load": 1, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "avg_session_uptime": 816, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "mdthresh_evicted": 0 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:37.231 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:37.232 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:37.233 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:37.234 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 283463, 2024-01-25T16:21:37.235 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 127, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:37.236 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 8352, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 190, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 153589, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 811, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:37.237 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "objecter": { 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "op_active": 0, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "op_laggy": 0, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send": 49, 2024-01-25T16:21:37.238 INFO:teuthology.orchestra.run.smithi152.stdout: "op_send_bytes": 13662, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "op_resend": 27, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "op_reply": 49, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "op_latency": { 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 49, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.331995981, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.006775428 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "op_inflight": 0, 2024-01-25T16:21:37.239 INFO:teuthology.orchestra.run.smithi152.stdout: "oplen_avg": { 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 49, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 72 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "op": 49, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "op_r": 0, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "op_w": 49, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "op_rmw": 0, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "op_pg": 0, 2024-01-25T16:21:37.240 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_stat": 0, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_create": 10, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_read": 0, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_write": 9, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writefull": 10, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_writesame": 0, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_append": 0, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_zero": 0, 2024-01-25T16:21:37.241 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_truncate": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_delete": 6, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_mapext": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_sparse_read": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_clonerange": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_getxattr": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_setxattr": 20, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_cmpxattr": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_rmxattr": 0, 2024-01-25T16:21:37.242 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_resetxattrs": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_call": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_watch": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_notify": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_src_cmpxattr": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_pgls_filter": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "osdop_other": 0, 2024-01-25T16:21:37.243 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_active": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_send": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_resend": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "linger_ping": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_active": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_send": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "poolop_resend": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_active": 0, 2024-01-25T16:21:37.244 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_send": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "poolstat_resend": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_active": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_send": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "statfs_resend": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "command_active": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "command_send": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "command_resend": 0, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "map_epoch": 112, 2024-01-25T16:21:37.245 INFO:teuthology.orchestra.run.smithi152.stdout: "map_full": 0, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "map_inc": 26, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_sessions": 9, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_open": 14, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_session_close": 5, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_laggy": 0, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_wr": 16, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_rd": 0, 2024-01-25T16:21:37.246 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_del": 1 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "oft": { 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_objs": 1, 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_kv_pairs": 0, 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_updates": 0, 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "omap_total_removes": 0 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "osd-slow-ops": { 2024-01-25T16:21:37.247 INFO:teuthology.orchestra.run.smithi152.stdout: "slow_ops_count": 0 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "purge_queue": { 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops": 0, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_ops_high_water": 0, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing": 0, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executing_high_water": 0, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed_ops": 0, 2024-01-25T16:21:37.248 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_executed": 0, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "pq_item_in_journal": 0 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 295605, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 117295299, 2024-01-25T16:21:37.249 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 295605, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 295605, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 117295299, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.250 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_bytes": { 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:37.251 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 49, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 13662, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 49, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 13662, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.252 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-objecter_ops": { 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 1024, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:37.253 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 49, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 49, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 49, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 49, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.254 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle": { 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 0, 2024-01-25T16:21:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.256 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-write_buf_throttle-0x5619e17ce6e0": { 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 3758096384, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 10, 2024-01-25T16:21:37.257 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 2879, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 10, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 9, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 2879, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:37.258 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:37.259 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.259 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:37.259 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-01-25T16:21:37.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:37 smithi033 ceph-mon[31417]: pgmap v439: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-01-25T16:21:37.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:37 smithi152 ceph-mon[36015]: pgmap v439: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-01-25T16:21:37.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:37 smithi076 ceph-mon[35642]: pgmap v439: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.8 MiB/s wr, 76 op/s 2024-01-25T16:21:38.039 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-01-25T16:21:38.039 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-01-25T16:21:38.039 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-01-25T16:21:38.040 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=326 2024-01-25T16:21:38.040 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-01-25T16:21:38.040 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-01-25T16:21:38.040 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-01-25T16:21:38.040 DEBUG:tasks.check_counter:Getting stats from l 2024-01-25T16:21:38.040 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5 shell --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 -- ceph daemon mds.l perf dump 2024-01-25T16:21:38.280 INFO:teuthology.orchestra.run.smithi152.stderr:Inferring config /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/mon.c/config 2024-01-25T16:21:38.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:38 smithi152 ceph-mon[36015]: pgmap v440: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-01-25T16:21:38.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:38 smithi076 ceph-mon[35642]: pgmap v440: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-01-25T16:21:38.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:38 smithi033 ceph-mon[31417]: pgmap v440: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-01-25T16:21:40.100 INFO:teuthology.orchestra.run.smithi152.stdout:{ 2024-01-25T16:21:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-0": { 2024-01-25T16:21:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 4, 2024-01-25T16:21:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 4, 2024-01-25T16:21:40.100 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 3040, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 448, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 4, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 0, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.001335797, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.000027951, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.000556550, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:40.101 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 3, 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000743459, 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000247819 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:40.102 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 448 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-1": { 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 2, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 202, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 320, 2024-01-25T16:21:40.103 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 189808, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 5, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 3, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.011303209, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.007516851, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.001441615, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:40.104 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 199, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012545290, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000063041 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.105 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 320, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 189808 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "AsyncMessenger::Worker-2": { 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_messages": 275, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages": 264, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_bytes": 1574528, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_bytes": 158976, 2024-01-25T16:21:40.106 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_created_connections": 2, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_active_connections": 1, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_total_time": 0.048646322, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_send_time": 0.014570790, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_recv_time": 0.037370918, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_running_fast_dispatch_time": 0.005651900, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_messages_queue_lat": { 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 263, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.012470552, 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000047416 2024-01-25T16:21:40.107 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_handle_ack_lat": { 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_recv_encrypted_bytes": 1574528, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "msgr_send_encrypted_bytes": 158976 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.108 INFO:teuthology.orchestra.run.smithi152.stdout: "cct": { 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "total_workers": 0, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "unhealthy_workers": 0 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "mempool": { 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_bytes": 0, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "bloom_filter_items": 0, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_bytes": 0, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_alloc_items": 0, 2024-01-25T16:21:40.109 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_bytes": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_data_items": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_bytes": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_onode_items": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_bytes": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_meta_items": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_bytes": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_other_items": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_bytes": 0, 2024-01-25T16:21:40.110 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_cache_buffer_items": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_bytes": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_extent_items": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_bytes": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_blob_items": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_bytes": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_shared_blob_items": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_bytes": 0, 2024-01-25T16:21:40.111 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_inline_bl_items": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_bytes": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_fsck_items": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_bytes": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_txc_items": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_bytes": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_deferred_items": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_bytes": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluestore_writing_items": 0, 2024-01-25T16:21:40.112 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_bytes": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_items": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_bytes": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_reader_items": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_bytes": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "bluefs_file_writer_items": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_bytes": 149007, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_anon_items": 30, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_bytes": 0, 2024-01-25T16:21:40.113 INFO:teuthology.orchestra.run.smithi152.stdout: "buffer_meta_items": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_bytes": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_items": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_bytes": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_mapbl_items": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_bytes": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osd_pglog_items": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_bytes": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_items": 0, 2024-01-25T16:21:40.114 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_bytes": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "osdmap_mapping_items": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_bytes": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "pgmap_items": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_bytes": 928, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "mds_co_items": 2, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_bytes": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_1_items": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_bytes": 0, 2024-01-25T16:21:40.115 INFO:teuthology.orchestra.run.smithi152.stdout: "unittest_2_items": 0 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: }, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "val": 0, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "max": 104857600, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "get_started": 0, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "get": 277, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "get_sum": 1535730, 2024-01-25T16:21:40.116 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_fail": 0, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "get_or_fail_success": 277, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "take": 0, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "take_sum": 0, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "put": 277, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "put_sum": 1535730, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "wait": { 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "avgcount": 0, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "sum": 0.000000000, 2024-01-25T16:21:40.117 INFO:teuthology.orchestra.run.smithi152.stdout: "avgtime": 0.000000000 2024-01-25T16:21:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:40.118 INFO:teuthology.orchestra.run.smithi152.stdout: } 2024-01-25T16:21:40.118 INFO:teuthology.orchestra.run.smithi152.stdout:} 2024-01-25T16:21:40.773 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-01-25T16:21:40.773 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-01-25T16:21:40.773 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-01-25T16:21:40.773 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-01-25T16:21:40.774 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-01-25T16:21:40.774 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-01-25T16:21:40.774 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-01-25T16:21:40.774 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-25T16:21:40.785 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-25T16:21:40.785 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T16:21:40.785 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-25T16:21:40.805 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-01-25T16:21:40.805 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-25T16:21:40.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:21:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 -- 172.21.15.33:0/2146221091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c0101f20 msgr2=0x7f56c0102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 --2- 172.21.15.33:0/2146221091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c0101f20 0x7f56c0102300 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009900 tx=0x7f56bc02bcd0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 -- 172.21.15.33:0/2146221091 shutdown_connections 2024-01-25T16:21:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 --2- 172.21.15.33:0/2146221091 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c010b2f0 0x7f56c010d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 --2- 172.21.15.33:0/2146221091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 0x7f56c010ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 --2- 172.21.15.33:0/2146221091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c0101f20 0x7f56c0102300 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 -- 172.21.15.33:0/2146221091 >> 172.21.15.33:0/2146221091 conn(0x7f56c0075f70 msgr2=0x7f56c0076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 -- 172.21.15.33:0/2146221091 shutdown_connections 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.022+0000 7f56c809b640 1 -- 172.21.15.33:0/2146221091 wait complete. 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 Processor -- start 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 -- start start 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c0101f20 0x7f56c0197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 0x7f56c0197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019c100 con 0x7f56c010b2f0 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019c270 con 0x7f56c0102840 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c809b640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c019c430 con 0x7f56c0101f20 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c6638640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c6638640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60074/0 (socket says 172.21.15.33:60074) 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c6638640 1 -- 172.21.15.33:0/3602604559 learned_addr learned my addr 172.21.15.33:0/3602604559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.023+0000 7f56c5636640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 0x7f56c0197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c5e37640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c0101f20 0x7f56c0197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 -- 172.21.15.33:0/3602604559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c0101f20 msgr2=0x7f56c0197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c0101f20 0x7f56c0197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 -- 172.21.15.33:0/3602604559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 msgr2=0x7f56c0197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 0x7f56c0197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 -- 172.21.15.33:0/3602604559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc009560 con 0x7f56c010b2f0 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c6638640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cab0 tx=0x7f56b800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b800ae20 con 0x7f56c010b2f0 2024-01-25T16:21:41.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b800f910 con 0x7f56c010b2f0 2024-01-25T16:21:41.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c019c710 con 0x7f56c010b2f0 2024-01-25T16:21:41.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80189a0 con 0x7f56c010b2f0 2024-01-25T16:21:41.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.024+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c01a3f90 con 0x7f56c010b2f0 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.025+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c0069b40 con 0x7f56c010b2f0 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.026+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f56b8018b40 con 0x7f56c010b2f0 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.027+0000 7f56b2ffd640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 0x7f56ac0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.027+0000 7f56c5e37640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 0x7f56ac0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.027+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f56b80cd3a0 con 0x7f56c010b2f0 2024-01-25T16:21:41.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.027+0000 7f56c5e37640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 0x7f56ac0a3910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc0046c0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.030+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f56b8096530 con 0x7f56c010b2f0 2024-01-25T16:21:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.213+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56c0106470 con 0x7f56c010b2f0 2024-01-25T16:21:41.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.214+0000 7f56b2ffd640 1 -- 172.21.15.33:0/3602604559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+22978 (secure 0 0 0) 0x7f56c0106470 con 0x7f56c010b2f0 2024-01-25T16:21:41.213 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:41.213 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":112,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:21:10.273290+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1587480808":"2024-01-25T17:12:43.374402+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2511336685":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/69013124":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/679037113":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/1063041969":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/441508626":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1063041969":"2024-01-26T16:07:50.056976+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:21:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 msgr2=0x7f56ac0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 0x7f56ac0a3910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f56bc000c00 tx=0x7f56bc0046c0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 msgr2=0x7f56c019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f56b800cab0 tx=0x7f56b800cf70 comp rx=0 tx=0).stop 2024-01-25T16:21:41.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 shutdown_connections 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f56ac0a1450 0x7f56ac0a3910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56c010b2f0 0x7f56c019bb00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56c0102840 0x7f56c0197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 --2- 172.21.15.33:0/3602604559 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56c0101f20 0x7f56c0197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.216+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 >> 172.21.15.33:0/3602604559 conn(0x7f56c0075f70 msgr2=0x7f56c00fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.217+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 shutdown_connections 2024-01-25T16:21:41.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.217+0000 7f56c809b640 1 -- 172.21.15.33:0/3602604559 wait complete. 2024-01-25T16:21:41.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-25T16:01:16.360313+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-25T16:05:44.202709+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-25T16:05:45.323169+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 111, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-25T16:21:41.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-25T16:21:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- 172.21.15.33:0/3725583655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0901018a0 msgr2=0x7fd0901058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 --2- 172.21.15.33:0/3725583655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0901018a0 0x7fd0901058e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd078009930 tx=0x7fd07802bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- 172.21.15.33:0/3725583655 shutdown_connections 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 --2- 172.21.15.33:0/3725583655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090105fc0 0x7fd09010db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 --2- 172.21.15.33:0/3725583655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd0901018a0 0x7fd0901058e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 --2- 172.21.15.33:0/3725583655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090100ef0 0x7fd0901012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- 172.21.15.33:0/3725583655 >> 172.21.15.33:0/3725583655 conn(0x7fd0900780f0 msgr2=0x7fd090078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- 172.21.15.33:0/3725583655 shutdown_connections 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- 172.21.15.33:0/3725583655 wait complete. 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 Processor -- start 2024-01-25T16:21:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.335+0000 7fd0955d7640 1 -- start start 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090100ef0 0x7fd090197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090105fc0 0x7fd0901977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019c510 con 0x7fd09019bb20 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019c680 con 0x7fd090105fc0 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09019c7f0 con 0x7fd090100ef0 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60094/0 (socket says 172.21.15.33:60094) 2024-01-25T16:21:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 -- 172.21.15.33:0/4248588580 learned_addr learned my addr 172.21.15.33:0/4248588580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08effd640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090100ef0 0x7fd090197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08e7fc640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090105fc0 0x7fd0901977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 -- 172.21.15.33:0/4248588580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090100ef0 msgr2=0x7fd090197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090100ef0 0x7fd090197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 -- 172.21.15.33:0/4248588580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090105fc0 msgr2=0x7fd0901977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090105fc0 0x7fd0901977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 -- 172.21.15.33:0/4248588580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078009560 con 0x7fd09019bb20 2024-01-25T16:21:41.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd08f7fe640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd08000eaa0 tx=0x7fd08000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08000cdb0 con 0x7fd09019bb20 2024-01-25T16:21:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd080005020 con 0x7fd09019bb20 2024-01-25T16:21:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0901a3dd0 con 0x7fd09019bb20 2024-01-25T16:21:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0800116d0 con 0x7fd09019bb20 2024-01-25T16:21:41.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.336+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0901a4200 con 0x7fd09019bb20 2024-01-25T16:21:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.337+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd090067f30 con 0x7fd09019bb20 2024-01-25T16:21:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.339+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd080005190 con 0x7fd09019bb20 2024-01-25T16:21:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.340+0000 7fd087fff640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 0x7fd07c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.340+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd0800cd660 con 0x7fd09019bb20 2024-01-25T16:21:41.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.340+0000 7fd08effd640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 0x7fd07c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.340+0000 7fd08effd640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 0x7fd07c0a3910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd088009b60 tx=0x7fd088005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.343+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd080096700 con 0x7fd09019bb20 2024-01-25T16:21:41.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.524+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0901018a0 con 0x7fd09019bb20 2024-01-25T16:21:41.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.524+0000 7fd087fff640 1 -- 172.21.15.33:0/4248588580 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd080099d20 con 0x7fd09019bb20 2024-01-25T16:21:41.523 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-01-25T16:21:41.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 msgr2=0x7fd07c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 0x7fd07c0a3910 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd088009b60 tx=0x7fd088005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:41.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 msgr2=0x7fd09019bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd08000eaa0 tx=0x7fd08000ef60 comp rx=0 tx=0).stop 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 shutdown_connections 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd07c0a1450 0x7fd07c0a3910 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd09019bb20 0x7fd09019bfd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd090105fc0 0x7fd0901977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 --2- 172.21.15.33:0/4248588580 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd090100ef0 0x7fd090197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.526+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 >> 172.21.15.33:0/4248588580 conn(0x7fd0900780f0 msgr2=0x7fd0900ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.527+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 shutdown_connections 2024-01-25T16:21:41.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.527+0000 7fd0955d7640 1 -- 172.21.15.33:0/4248588580 wait complete. 2024-01-25T16:21:41.534 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:41 smithi033 ceph-mon[31417]: pgmap v441: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-01-25T16:21:41.535 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:41 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3602604559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:41.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-25T16:21:41.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:41 smithi152 ceph-mon[36015]: pgmap v441: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-01-25T16:21:41.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:41 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3602604559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 -- 172.21.15.33:0/2693914210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 msgr2=0x7f691810f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 --2- 172.21.15.33:0/2693914210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691810f930 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6910009900 tx=0x7f691002bcd0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 -- 172.21.15.33:0/2693914210 shutdown_connections 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 --2- 172.21.15.33:0/2693914210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691810f930 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 --2- 172.21.15.33:0/2693914210 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69180739c0 0x7f691810d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 --2- 172.21.15.33:0/2693914210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180730a0 0x7f6918073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.642+0000 7f691e087640 1 -- 172.21.15.33:0/2693914210 >> 172.21.15.33:0/2693914210 conn(0x7f69180fc790 msgr2=0x7f69180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- 172.21.15.33:0/2693914210 shutdown_connections 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- 172.21.15.33:0/2693914210 wait complete. 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 Processor -- start 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- start start 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69180730a0 0x7f69181994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180739c0 0x7f69181999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819e310 con 0x7f691810d540 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819e480 con 0x7f69180739c0 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.643+0000 7f691e087640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691819e640 con 0x7f69180730a0 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60110/0 (socket says 172.21.15.33:60110) 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 -- 172.21.15.33:0/2349088220 learned_addr learned my addr 172.21.15.33:0/2349088220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6916ffd640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180739c0 0x7f69181999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 -- 172.21.15.33:0/2349088220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69180730a0 msgr2=0x7f69181994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69180730a0 0x7f69181994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 -- 172.21.15.33:0/2349088220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180739c0 msgr2=0x7f69181999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180739c0 0x7f69181999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 -- 172.21.15.33:0/2349088220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009560 con 0x7f691810d540 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6917fff640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6910009230 tx=0x7f6910004dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691002dac0 con 0x7f691810d540 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69100054c0 con 0x7f691810d540 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691819e8c0 con 0x7f691810d540 2024-01-25T16:21:41.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6910003af0 con 0x7f691810d540 2024-01-25T16:21:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.644+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69181a61a0 con 0x7f691810d540 2024-01-25T16:21:41.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.645+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6918067ae0 con 0x7f691810d540 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.646+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f691002dc20 con 0x7f691810d540 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.647+0000 7f6914ff9640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 0x7f69040a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.647+0000 7f69177fe640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 0x7f69040a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.647+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f69100efa40 con 0x7f691810d540 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.647+0000 7f69177fe640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 0x7f69040a3910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6908005fd0 tx=0x7f6908009440 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.650+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f69100b8bf0 con 0x7f691810d540 2024-01-25T16:21:41.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:41 smithi076 ceph-mon[35642]: pgmap v441: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-01-25T16:21:41.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:41 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3602604559' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:41.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.831+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6918108750 con 0x7f691810d540 2024-01-25T16:21:41.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.831+0000 7f6914ff9640 1 -- 172.21.15.33:0/2349088220 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v112) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6918108750 con 0x7f691810d540 2024-01-25T16:21:41.830 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 msgr2=0x7f69040a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 0x7f69040a3910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6908005fd0 tx=0x7f6908009440 comp rx=0 tx=0).stop 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 msgr2=0x7f691819dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6910009230 tx=0x7f6910004dd0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 shutdown_connections 2024-01-25T16:21:41.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f69040a1450 0x7f69040a3910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f691810d540 0x7f691819dd10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69180739c0 0x7f69181999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 --2- 172.21.15.33:0/2349088220 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f69180730a0 0x7f69181994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 >> 172.21.15.33:0/2349088220 conn(0x7f69180fc790 msgr2=0x7f69180fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 shutdown_connections 2024-01-25T16:21:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.833+0000 7f691e087640 1 -- 172.21.15.33:0/2349088220 wait complete. 2024-01-25T16:21:41.843 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-25T16:21:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/2191283320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc100790 msgr2=0x7fb9dc1047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/2191283320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc100790 0x7fb9dc1047d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009930 tx=0x7fb9c802bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/2191283320 shutdown_connections 2024-01-25T16:21:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/2191283320 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc10ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/2191283320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc100790 0x7fb9dc1047d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/2191283320 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/2191283320 >> 172.21.15.33:0/2191283320 conn(0x7fb9dc075f70 msgr2=0x7fb9dc076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/2191283320 shutdown_connections 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/2191283320 wait complete. 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 Processor -- start 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.950+0000 7fb9e2bd5640 1 -- start start 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 0x7fb9dc19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19c620 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19c790 con 0x7fb9dc19bc30 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9dc19c900 con 0x7fb9dc104eb0 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60128/0 (socket says 172.21.15.33:60128) 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 -- 172.21.15.33:0/1016079479 learned_addr learned my addr 172.21.15.33:0/1016079479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:41.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e1172640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 0x7fb9dc19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 -- 172.21.15.33:0/1016079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 msgr2=0x7fb9dc1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9dbfff640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 -- 172.21.15.33:0/1016079479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 msgr2=0x7fb9dc19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 0x7fb9dc19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 -- 172.21.15.33:0/1016079479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8009560 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e1172640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 0x7fb9dc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:21:41.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9dbfff640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e0971640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e9f0 tx=0x7fb9d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0009700 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9d000cd80 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9dc1a3ee0 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.951+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0018860 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.952+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9dc1a4310 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.953+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9dc1012b0 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.954+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fb9d00189c0 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.954+0000 7fb9d9ffb640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 0x7fb9cc0a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:41.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.954+0000 7fb9dbfff640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 0x7fb9cc0a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.954+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb9d0014070 con 0x7fb9dc0ffde0 2024-01-25T16:21:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.955+0000 7fb9dbfff640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 0x7fb9cc0a3990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8006010 tx=0x7fb9c8000bc0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:41.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:41.959+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fb9d0052600 con 0x7fb9dc0ffde0 2024-01-25T16:21:42.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.137+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9dc100840 con 0x7fb9dc0ffde0 2024-01-25T16:21:42.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.138+0000 7fb9d9ffb640 1 -- 172.21.15.33:0/1016079479 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v112) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9d0096480 con 0x7fb9dc0ffde0 2024-01-25T16:21:42.137 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-01-25T16:21:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 msgr2=0x7fb9cc0a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 0x7fb9cc0a3990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8006010 tx=0x7fb9c8000bc0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 msgr2=0x7fb9dc197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e9f0 tx=0x7fb9d000eeb0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 shutdown_connections 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fb9cc0a14d0 0x7fb9cc0a3990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9dc19bc30 0x7fb9dc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb9dc104eb0 0x7fb9dc1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 --2- 172.21.15.33:0/1016079479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9dc0ffde0 0x7fb9dc197370 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 >> 172.21.15.33:0/1016079479 conn(0x7fb9dc075f70 msgr2=0x7fb9dc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 shutdown_connections 2024-01-25T16:21:42.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.140+0000 7fb9e2bd5640 1 -- 172.21.15.33:0/1016079479 wait complete. 2024-01-25T16:21:42.150 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-25T16:21:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.254+0000 7f258f368640 1 -- 172.21.15.33:0/3615596195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588100780 msgr2=0x7f25881047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.254+0000 7f258f368640 1 --2- 172.21.15.33:0/3615596195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588100780 0x7f25881047c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2578009930 tx=0x7f257802bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- 172.21.15.33:0/3615596195 shutdown_connections 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- 172.21.15.33:0/3615596195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 0x7f258810ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- 172.21.15.33:0/3615596195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2588100780 0x7f25881047c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- 172.21.15.33:0/3615596195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 0x7f25881001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- 172.21.15.33:0/3615596195 >> 172.21.15.33:0/3615596195 conn(0x7f2588076cf0 msgr2=0x7f2588077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- 172.21.15.33:0/3615596195 shutdown_connections 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- 172.21.15.33:0/3615596195 wait complete. 2024-01-25T16:21:42.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 Processor -- start 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- start start 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 0x7f25881972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 0x7f2588197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258819c520 con 0x7f258819bb30 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258819c690 con 0x7f25880ffdd0 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.255+0000 7f258f368640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258819c800 con 0x7f2588104ea0 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60136/0 (socket says 172.21.15.33:60136) 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 -- 172.21.15.33:0/3255652543 learned_addr learned my addr 172.21.15.33:0/3255652543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258c903640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 0x7f2588197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 -- 172.21.15.33:0/3255652543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 msgr2=0x7f2588197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d104640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 0x7f25881972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 0x7f2588197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 -- 172.21.15.33:0/3255652543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 msgr2=0x7f25881972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 0x7f25881972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 -- 172.21.15.33:0/3255652543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009560 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258d905640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f258000eaa0 tx=0x7f258000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258000cdb0 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2580005020 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25881a3de0 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25800116d0 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.256+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25881a4290 con 0x7f258819bb30 2024-01-25T16:21:42.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.257+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2588067ae0 con 0x7f258819bb30 2024-01-25T16:21:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.258+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f2580005190 con 0x7f258819bb30 2024-01-25T16:21:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.259+0000 7f257e7fc640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 0x7f25740a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.259+0000 7f258d104640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 0x7f25740a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.259+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f25800cd130 con 0x7f258819bb30 2024-01-25T16:21:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.259+0000 7f258d104640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 0x7f25740a3990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2584009aa0 tx=0x7f2584005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:42.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.263+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f25800962d0 con 0x7f258819bb30 2024-01-25T16:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.455+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25881985c0 con 0x7f258819bb30 2024-01-25T16:21:42.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.456+0000 7f257e7fc640 1 -- 172.21.15.33:0/3255652543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v112) v1 ==== 81+0+998 (secure 0 0 0) 0x7f258001f080 con 0x7f258819bb30 2024-01-25T16:21:42.454 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1587480808 2024-01-25T17:12:43.374402+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4223658721 2024-01-26T15:58:21.567761+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2511336685 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/123634986 2024-01-26T15:58:48.018885+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/69013124 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/679037113 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1043188550 2024-01-26T15:58:21.567761+0000 2024-01-25T16:21:42.455 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2443786608 2024-01-26T15:58:48.018885+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2443786608 2024-01-26T15:58:48.018885+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1043188550 2024-01-26T15:58:21.567761+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1063041969 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2874813062 2024-01-26T15:58:48.018885+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2680727410 2024-01-26T15:58:21.567761+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/797972698 2024-01-26T15:58:48.018885+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3011083041 2024-01-26T15:58:21.567761+0000 2024-01-25T16:21:42.456 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/441508626 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.457 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1063041969 2024-01-26T16:07:50.056976+0000 2024-01-25T16:21:42.457 INFO:teuthology.orchestra.run.smithi033.stderr:listed 17 entries 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 msgr2=0x7f25740a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 0x7f25740a3990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2584009aa0 tx=0x7f2584005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 msgr2=0x7f258819bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f258000eaa0 tx=0x7f258000ef60 comp rx=0 tx=0).stop 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 shutdown_connections 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25740a14d0 0x7f25740a3990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f258819bb30 0x7f258819bfe0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f2588104ea0 0x7f2588197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 --2- 172.21.15.33:0/3255652543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25880ffdd0 0x7f25881972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 >> 172.21.15.33:0/3255652543 conn(0x7f2588076cf0 msgr2=0x7f25880682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 shutdown_connections 2024-01-25T16:21:42.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.458+0000 7f258f368640 1 -- 172.21.15.33:0/3255652543 wait complete. 2024-01-25T16:21:42.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4248588580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/2349088220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.466 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:42 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1016079479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.467 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-25T16:21:42.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:21:42.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-25T16:21:42.561 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:21:42.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:21:42.561 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-25T16:21:42.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:21:42.605 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-01-25T16:21:42.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4248588580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/2349088220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:42 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1016079479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4248588580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/2349088220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:42 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1016079479' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-01-25T16:21:42.673 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-25T16:21:42.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:21:42.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-25T16:21:42.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-25T16:21:42.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-01-25T16:21:42.832 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-01-25T16:21:42.843 INFO:tasks.cephadm:Teardown begin 2024-01-25T16:21:42.844 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-25T16:21:42.844 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:21:42.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- 172.21.15.33:0/1683017880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a8106070 msgr2=0x7f25a810dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 --2- 172.21.15.33:0/1683017880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a8106070 0x7f25a810dbf0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f25a0009cd0 tx=0x7f25a002c0f0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- 172.21.15.33:0/1683017880 shutdown_connections 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 --2- 172.21.15.33:0/1683017880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a8106070 0x7f25a810dbf0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 --2- 172.21.15.33:0/1683017880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a80ffe10 0x7f25a8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 --2- 172.21.15.33:0/1683017880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a80ff460 0x7f25a80ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- 172.21.15.33:0/1683017880 >> 172.21.15.33:0/1683017880 conn(0x7f25a80761b0 msgr2=0x7f25a80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- 172.21.15.33:0/1683017880 shutdown_connections 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- 172.21.15.33:0/1683017880 wait complete. 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 Processor -- start 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.949+0000 7f25b0275640 1 -- start start 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a8106070 0x7f25a8100910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a8100e50 0x7f25a81a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81a5a40 con 0x7f25a80ff460 2024-01-25T16:21:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81a5bb0 con 0x7f25a8106070 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a81a5d70 con 0x7f25a8100e50 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60168/0 (socket says 172.21.15.33:60168) 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 -- 172.21.15.33:0/1930431593 learned_addr learned my addr 172.21.15.33:0/1930431593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae812640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a8100e50 0x7f25a81a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ad810640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a8106070 0x7f25a8100910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 -- 172.21.15.33:0/1930431593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a8100e50 msgr2=0x7f25a81a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a8100e50 0x7f25a81a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 -- 172.21.15.33:0/1930431593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a8106070 msgr2=0x7f25a8100910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a8106070 0x7f25a8100910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 -- 172.21.15.33:0/1930431593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0009900 con 0x7f25a80ff460 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25ae011640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9f0 tx=0x7f25a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4009720 con 0x7f25a80ff460 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25a400cd80 con 0x7f25a80ff460 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a81a5ff0 con 0x7f25a80ff460 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4010430 con 0x7f25a80ff460 2024-01-25T16:21:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.950+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a81a64d0 con 0x7f25a80ff460 2024-01-25T16:21:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.951+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a8067f30 con 0x7f25a80ff460 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.953+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f25a4005000 con 0x7f25a80ff460 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.953+0000 7f259f7fe640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 0x7f25940a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.953+0000 7f25ad810640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 0x7f25940a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.953+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f25a4014070 con 0x7f25a80ff460 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.954+0000 7f25ad810640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 0x7f25940a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25a8101e60 tx=0x7f2598005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:42.957+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f25a4096590 con 0x7f25a80ff460 2024-01-25T16:21:43.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.175+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25a8101900 con 0x7f25a80ff460 2024-01-25T16:21:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.175+0000 7f259f7fe640 1 -- 172.21.15.33:0/1930431593 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11337 (secure 0 0 0) 0x7f25a4094e60 con 0x7f25a80ff460 2024-01-25T16:21:43.174 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:43.174 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:20:33.324846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25982":{"gid":25982,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/689332184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":689332184},{"type":"v1","addr":"172.21.15.152:6837","nonce":689332184}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:21:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 43 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 msgr2=0x7f25940a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 0x7f25940a39c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25a8101e60 tx=0x7f2598005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 msgr2=0x7f25a81003d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f25a400e9f0 tx=0x7f25a400edf0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 shutdown_connections 2024-01-25T16:21:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f25940a1500 0x7f25940a39c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f25a8100e50 0x7f25a81a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f25a8106070 0x7f25a8100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 --2- 172.21.15.33:0/1930431593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25a80ff460 0x7f25a81003d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 >> 172.21.15.33:0/1930431593 conn(0x7f25a80761b0 msgr2=0x7f25a80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 shutdown_connections 2024-01-25T16:21:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.177+0000 7f25b0275640 1 -- 172.21.15.33:0/1930431593 wait complete. 2024-01-25T16:21:43.186 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-25T16:21:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 -- 172.21.15.33:0/1820519848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7330069fc0 msgr2=0x7f7330105930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 --2- 172.21.15.33:0/1820519848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7330069fc0 0x7f7330105930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7324009890 tx=0x7f732402bb70 comp rx=0 tx=0).stop 2024-01-25T16:21:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 -- 172.21.15.33:0/1820519848 shutdown_connections 2024-01-25T16:21:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 --2- 172.21.15.33:0/1820519848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330106010 0x7f733010dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 --2- 172.21.15.33:0/1820519848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7330069fc0 0x7f7330105930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 --2- 172.21.15.33:0/1820519848 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330069610 0x7f73300699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 -- 172.21.15.33:0/1820519848 >> 172.21.15.33:0/1820519848 conn(0x7f7330076c90 msgr2=0x7f73300770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 -- 172.21.15.33:0/1820519848 shutdown_connections 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.293+0000 7f7334d2f640 1 -- 172.21.15.33:0/1820519848 wait complete. 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 Processor -- start 2024-01-25T16:21:43.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 -- start start 2024-01-25T16:21:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330106010 0x7f73301977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 0x7f733019c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733019c560 con 0x7f7330069610 2024-01-25T16:21:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733019c6d0 con 0x7f733019bb70 2024-01-25T16:21:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f7334d2f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733019c840 con 0x7f7330106010 2024-01-25T16:21:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60186/0 (socket says 172.21.15.33:60186) 2024-01-25T16:21:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 -- 172.21.15.33:0/4292464141 learned_addr learned my addr 172.21.15.33:0/4292464141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:43.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732dd9b640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330106010 0x7f73301977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732ed9d640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 0x7f733019c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 -- 172.21.15.33:0/4292464141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330106010 msgr2=0x7f73301977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330106010 0x7f73301977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 -- 172.21.15.33:0/4292464141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 msgr2=0x7f733019c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 0x7f733019c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732e59c640 1 -- 172.21.15.33:0/4292464141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7328009560 con 0x7f7330069610 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.294+0000 7f732ed9d640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 0x7f733019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f732e59c640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f732000eab0 tx=0x7f732000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732000ce40 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7324009490 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7320011930 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320010a30 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.295+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73301a4110 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.296+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7330102410 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.297+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f7320011470 con 0x7f7330069610 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.297+0000 7f731f7fe640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 0x7f73180a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.297+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f73200cd540 con 0x7f7330069610 2024-01-25T16:21:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.297+0000 7f732dd9b640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 0x7f73180a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.298+0000 7f732dd9b640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 0x7f73180a3910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f732402cf50 tx=0x7f732402df90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.301+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f7320096640 con 0x7f7330069610 2024-01-25T16:21:43.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.516+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7330067f30 con 0x7f7330069610 2024-01-25T16:21:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.517+0000 7f731f7fe640 1 -- 172.21.15.33:0/4292464141 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+11337 (secure 0 0 0) 0x7f7320099c60 con 0x7f7330069610 2024-01-25T16:21:43.516 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:43.516 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14562,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2241105736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":2241105736},{"type":"v1","addr":"172.21.15.33:6835","nonce":2241105736}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/2646618640","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":2646618640},{"type":"v1","addr":"172.21.15.33:6837","nonce":2646618640}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14637,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/3604277754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":3604277754},{"type":"v1","addr":"172.21.15.33:6839","nonce":3604277754}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14649,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/3585610078","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":3585610078},{"type":"v1","addr":"172.21.15.33:6841","nonce":3585610078}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24424,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6833/3591759228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3591759228},{"type":"v1","addr":"172.21.15.152:6833","nonce":3591759228}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24437,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6833/3364885054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3364885054},{"type":"v1","addr":"172.21.15.76:6833","nonce":3364885054}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-01-25T16:05:46.107985+0000","modified":"2024-01-25T16:20:33.324846+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24536,"mds_1":24517,"mds_2":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24464":{"gid":24464,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.76:6835/3469230010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3469230010},{"type":"v1","addr":"172.21.15.76:6835","nonce":3469230010}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24484":{"gid":24484,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6835/1042830191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":1042830191},{"type":"v1","addr":"172.21.15.152:6835","nonce":1042830191}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24494":{"gid":24494,"name":"h","rank":2,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6837/2322550881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2322550881},{"type":"v1","addr":"172.21.15.76:6837","nonce":2322550881}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24517":{"gid":24517,"name":"i","rank":1,"incarnation":17,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6839/2298219777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":2298219777},{"type":"v1","addr":"172.21.15.152:6839","nonce":2298219777}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24536":{"gid":24536,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6839/406718297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6838","nonce":406718297},{"type":"v1","addr":"172.21.15.76:6839","nonce":406718297}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_25982":{"gid":25982,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.152:6837/689332184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":689332184},{"type":"v1","addr":"172.21.15.152:6837","nonce":689332184}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-25T16:21:43.516 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 43 2024-01-25T16:21:43.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 msgr2=0x7f73180a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 0x7f73180a3910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f732402cf50 tx=0x7f732402df90 comp rx=0 tx=0).stop 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 msgr2=0x7f73301972b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f732000eab0 tx=0x7f732000ef70 comp rx=0 tx=0).stop 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 shutdown_connections 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f73180a1450 0x7f73180a3910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f733019bb70 0x7f733019c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7330106010 0x7f73301977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 --2- 172.21.15.33:0/4292464141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7330069610 0x7f73301972b0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 >> 172.21.15.33:0/4292464141 conn(0x7f7330076c90 msgr2=0x7f73300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 shutdown_connections 2024-01-25T16:21:43.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.519+0000 7f7334d2f640 1 -- 172.21.15.33:0/4292464141 wait complete. 2024-01-25T16:21:43.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:43 smithi033 ceph-mon[31417]: pgmap v442: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-01-25T16:21:43.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3255652543' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-25T16:21:43.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:43 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/1930431593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:43.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-25T16:21:43.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:43 smithi152 ceph-mon[36015]: pgmap v442: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-01-25T16:21:43.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3255652543' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-25T16:21:43.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:43 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/1930431593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 -- 172.21.15.33:0/2844557549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f600fff40 msgr2=0x7f6f60103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 --2- 172.21.15.33:0/2844557549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f600fff40 0x7f6f60103f80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009930 tx=0x7f6f4c02bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 -- 172.21.15.33:0/2844557549 shutdown_connections 2024-01-25T16:21:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 --2- 172.21.15.33:0/2844557549 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f60104660 0x7f6f6010c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 --2- 172.21.15.33:0/2844557549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f600fff40 0x7f6f60103f80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 --2- 172.21.15.33:0/2844557549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f600ff590 0x7f6f600ff970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 -- 172.21.15.33:0/2844557549 >> 172.21.15.33:0/2844557549 conn(0x7f6f60076180 msgr2=0x7f6f600fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 -- 172.21.15.33:0/2844557549 shutdown_connections 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.633+0000 7f6f67256640 1 -- 172.21.15.33:0/2844557549 wait complete. 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 Processor -- start 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 -- start start 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 0x7f6f60198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 0x7f6f601986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f601a3890 con 0x7f6f60198c40 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f601a3a00 con 0x7f6f60104660 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f67256640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f601a3bc0 con 0x7f6f600ff590 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60202/0 (socket says 172.21.15.33:60202) 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 -- 172.21.15.33:0/54163334 learned_addr learned my addr 172.21.15.33:0/54163334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:43.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f64ff2640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 0x7f6f60198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f5ffff640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 0x7f6f601986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 -- 172.21.15.33:0/54163334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 msgr2=0x7f6f60198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 0x7f6f60198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 -- 172.21.15.33:0/54163334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 msgr2=0x7f6f601986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 0x7f6f601986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 -- 172.21.15.33:0/54163334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009560 con 0x7f6f60198c40 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f5ffff640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 0x7f6f601986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:43.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f64ff2640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 0x7f6f60198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.634+0000 7f6f657f3640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009fd0 tx=0x7f6f5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.635+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800bea0 con 0x7f6f60198c40 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.635+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f58009940 con 0x7f6f60198c40 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.635+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f601a3ea0 con 0x7f6f60198c40 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.635+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f580106c0 con 0x7f6f60198c40 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.635+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f601a4380 con 0x7f6f60198c40 2024-01-25T16:21:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.636+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f60067f30 con 0x7f6f60198c40 2024-01-25T16:21:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.637+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f6f580041b0 con 0x7f6f60198c40 2024-01-25T16:21:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.637+0000 7f6f5dffb640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 0x7f6f500a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.637+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f580ccc00 con 0x7f6f60198c40 2024-01-25T16:21:43.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.637+0000 7f6f64ff2640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 0x7f6f500a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.638+0000 7f6f64ff2640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 0x7f6f500a3990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f540058c0 tx=0x7f6f54005850 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.642+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f6f580d3080 con 0x7f6f60198c40 2024-01-25T16:21:43.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:43 smithi076 ceph-mon[35642]: pgmap v442: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-01-25T16:21:43.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3255652543' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-01-25T16:21:43.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:43 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/1930431593' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.824+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f600fff40 con 0x7f6f60198c40 2024-01-25T16:21:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.825+0000 7f6f5dffb640 1 -- 172.21.15.33:0/54163334 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+22978 (secure 0 0 0) 0x7f6f58095d00 con 0x7f6f60198c40 2024-01-25T16:21:43.824 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-01-25T16:21:43.825 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":112,"fsid":"55cf2542-bb9a-11ee-95b2-87774f69a715","created":"2024-01-25T15:57:54.453261+0000","modified":"2024-01-25T16:21:10.273290+0000","last_up_change":"2024-01-25T16:04:55.827448+0000","last_in_change":"2024-01-25T16:04:35.833045+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-25T16:01:16.360313+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-25T16:05:44.202709+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-25T16:05:45.323169+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":23,"snap_epoch":111,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"83211537-b4b8-4002-921b-1a85501279cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6803","nonce":229778098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6805","nonce":229778098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6809","nonce":229778098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":229778098},{"type":"v1","addr":"172.21.15.33:6807","nonce":229778098}]},"public_addr":"172.21.15.33:6803/229778098","cluster_addr":"172.21.15.33:6805/229778098","heartbeat_back_addr":"172.21.15.33:6809/229778098","heartbeat_front_addr":"172.21.15.33:6807/229778098","state":["exists","up"]},{"osd":1,"uuid":"c90ee0f1-8aa4-48ce-8068-df0d16d4c86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6801","nonce":2892864635}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6803","nonce":2892864635}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6807","nonce":2892864635}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2892864635},{"type":"v1","addr":"172.21.15.76:6805","nonce":2892864635}]},"public_addr":"172.21.15.76:6801/2892864635","cluster_addr":"172.21.15.76:6803/2892864635","heartbeat_back_addr":"172.21.15.76:6807/2892864635","heartbeat_front_addr":"172.21.15.76:6805/2892864635","state":["exists","up"]},{"osd":2,"uuid":"337d041f-de02-49ec-87c1-d9e211edacff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6801","nonce":1513420968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6803","nonce":1513420968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6807","nonce":1513420968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":1513420968},{"type":"v1","addr":"172.21.15.152:6805","nonce":1513420968}]},"public_addr":"172.21.15.152:6801/1513420968","cluster_addr":"172.21.15.152:6803/1513420968","heartbeat_back_addr":"172.21.15.152:6807/1513420968","heartbeat_front_addr":"172.21.15.152:6805/1513420968","state":["exists","up"]},{"osd":3,"uuid":"d47d7676-603d-43dd-84d5-278ad2c00170","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6811","nonce":3993087703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6813","nonce":3993087703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6817","nonce":3993087703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":3993087703},{"type":"v1","addr":"172.21.15.33:6815","nonce":3993087703}]},"public_addr":"172.21.15.33:6811/3993087703","cluster_addr":"172.21.15.33:6813/3993087703","heartbeat_back_addr":"172.21.15.33:6817/3993087703","heartbeat_front_addr":"172.21.15.33:6815/3993087703","state":["exists","up"]},{"osd":4,"uuid":"83d9eb98-5eb6-40d7-b0d3-2651cc0b8bcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6809","nonce":1997772054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6811","nonce":1997772054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6815","nonce":1997772054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1997772054},{"type":"v1","addr":"172.21.15.76:6813","nonce":1997772054}]},"public_addr":"172.21.15.76:6809/1997772054","cluster_addr":"172.21.15.76:6811/1997772054","heartbeat_back_addr":"172.21.15.76:6815/1997772054","heartbeat_front_addr":"172.21.15.76:6813/1997772054","state":["exists","up"]},{"osd":5,"uuid":"722b8a67-2c0c-46f4-bcac-37587042f804","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6809","nonce":984769495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6811","nonce":984769495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6815","nonce":984769495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":984769495},{"type":"v1","addr":"172.21.15.152:6813","nonce":984769495}]},"public_addr":"172.21.15.152:6809/984769495","cluster_addr":"172.21.15.152:6811/984769495","heartbeat_back_addr":"172.21.15.152:6815/984769495","heartbeat_front_addr":"172.21.15.152:6813/984769495","state":["exists","up"]},{"osd":6,"uuid":"5c38c5ba-1402-4433-9924-49b144ff520d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6819","nonce":2008861778}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6821","nonce":2008861778}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6825","nonce":2008861778}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2008861778},{"type":"v1","addr":"172.21.15.33:6823","nonce":2008861778}]},"public_addr":"172.21.15.33:6819/2008861778","cluster_addr":"172.21.15.33:6821/2008861778","heartbeat_back_addr":"172.21.15.33:6825/2008861778","heartbeat_front_addr":"172.21.15.33:6823/2008861778","state":["exists","up"]},{"osd":7,"uuid":"dc7c7fb6-7719-4f21-8ef5-103d43fa1a49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6817","nonce":610247607}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6819","nonce":610247607}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6823","nonce":610247607}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":610247607},{"type":"v1","addr":"172.21.15.76:6821","nonce":610247607}]},"public_addr":"172.21.15.76:6817/610247607","cluster_addr":"172.21.15.76:6819/610247607","heartbeat_back_addr":"172.21.15.76:6823/610247607","heartbeat_front_addr":"172.21.15.76:6821/610247607","state":["exists","up"]},{"osd":8,"uuid":"aedc05dd-ab90-4ad9-9937-a3a7b56d4f72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6817","nonce":281006087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6819","nonce":281006087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6823","nonce":281006087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":281006087},{"type":"v1","addr":"172.21.15.152:6821","nonce":281006087}]},"public_addr":"172.21.15.152:6817/281006087","cluster_addr":"172.21.15.152:6819/281006087","heartbeat_back_addr":"172.21.15.152:6823/281006087","heartbeat_front_addr":"172.21.15.152:6821/281006087","state":["exists","up"]},{"osd":9,"uuid":"57cd9e56-03a7-4be8-98c0-27b71dadb5a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6827","nonce":877151460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6829","nonce":877151460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6833","nonce":877151460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":877151460},{"type":"v1","addr":"172.21.15.33:6831","nonce":877151460}]},"public_addr":"172.21.15.33:6827/877151460","cluster_addr":"172.21.15.33:6829/877151460","heartbeat_back_addr":"172.21.15.33:6833/877151460","heartbeat_front_addr":"172.21.15.33:6831/877151460","state":["exists","up"]},{"osd":10,"uuid":"06adc495-98fd-4870-97cf-0cd51993ca94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6825","nonce":772498776}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6827","nonce":772498776}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6831","nonce":772498776}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":772498776},{"type":"v1","addr":"172.21.15.76:6829","nonce":772498776}]},"public_addr":"172.21.15.76:6825/772498776","cluster_addr":"172.21.15.76:6827/772498776","heartbeat_back_addr":"172.21.15.76:6831/772498776","heartbeat_front_addr":"172.21.15.76:6829/772498776","state":["exists","up"]},{"osd":11,"uuid":"bd65bf47-57d6-49f1-ac41-49ef945e9bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6825","nonce":249910241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6827","nonce":249910241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6831","nonce":249910241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":249910241},{"type":"v1","addr":"172.21.15.152:6829","nonce":249910241}]},"public_addr":"172.21.15.152:6825/249910241","cluster_addr":"172.21.15.152:6827/249910241","heartbeat_back_addr":"172.21.15.152:6831/249910241","heartbeat_front_addr":"172.21.15.152:6829/249910241","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:24.707416+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:00:48.775754+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:11.710069+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:35.269198+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:01:59.839005+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:23.779342+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:02:48.557460+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:13.526088+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:03:37.411087+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:03.272200+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:28.738683+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-25T16:04:53.851093+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.e","mappings":[{"from":4,"to":8}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/1587480808":"2024-01-25T17:12:43.374402+0000","172.21.15.33:0/4223658721":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/2511336685":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/123634986":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/69013124":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/679037113":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6801/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/2443786608":"2024-01-26T15:58:48.018885+0000","172.21.15.33:6800/1043188550":"2024-01-26T15:58:21.567761+0000","172.21.15.33:6800/1063041969":"2024-01-26T16:07:50.056976+0000","172.21.15.33:0/2874813062":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/2680727410":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/797972698":"2024-01-26T15:58:48.018885+0000","172.21.15.33:0/3011083041":"2024-01-26T15:58:21.567761+0000","172.21.15.33:0/441508626":"2024-01-26T16:07:50.056976+0000","172.21.15.33:6801/1063041969":"2024-01-26T16:07:50.056976+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":10,"length":1},{"begin":23,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-25T16:21:43.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 msgr2=0x7f6f500a3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 0x7f6f500a3990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6f540058c0 tx=0x7f6f54005850 comp rx=0 tx=0).stop 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 msgr2=0x7f6f601a31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009fd0 tx=0x7f6f5800b260 comp rx=0 tx=0).stop 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 shutdown_connections 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f6f500a14d0 0x7f6f500a3990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f60198c40 0x7f6f601a31f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f60104660 0x7f6f601986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 --2- 172.21.15.33:0/54163334 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f600ff590 0x7f6f60198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 >> 172.21.15.33:0/54163334 conn(0x7f6f60076180 msgr2=0x7f6f600fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 shutdown_connections 2024-01-25T16:21:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.828+0000 7f6f67256640 1 -- 172.21.15.33:0/54163334 wait complete. 2024-01-25T16:21:43.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-25T16:21:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 -- 172.21.15.33:0/1512128685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881007c0 msgr2=0x7f1488104800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 --2- 172.21.15.33:0/1512128685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881007c0 0x7f1488104800 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1470009930 tx=0x7f147002bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 -- 172.21.15.33:0/1512128685 shutdown_connections 2024-01-25T16:21:43.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 --2- 172.21.15.33:0/1512128685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1488104ee0 0x7f148810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 --2- 172.21.15.33:0/1512128685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14881007c0 0x7f1488104800 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 --2- 172.21.15.33:0/1512128685 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f14880ffe10 0x7f14881001f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 -- 172.21.15.33:0/1512128685 >> 172.21.15.33:0/1512128685 conn(0x7f1488075f70 msgr2=0x7f1488076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 -- 172.21.15.33:0/1512128685 shutdown_connections 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.942+0000 7f148d8ee640 1 -- 172.21.15.33:0/1512128685 wait complete. 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 Processor -- start 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 -- start start 2024-01-25T16:21:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14880ffe10 0x7f14881972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1488104ee0 0x7f1488197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819c5a0 con 0x7f148819bbb0 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819c710 con 0x7f14880ffe10 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f148d8ee640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148819c880 con 0x7f1488104ee0 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f14877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60222/0 (socket says 172.21.15.33:60222) 2024-01-25T16:21:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f14877fe640 1 -- 172.21.15.33:0/4128629286 learned_addr learned my addr 172.21.15.33:0/4128629286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.943+0000 7f1486ffd640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14880ffe10 0x7f14881972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f147ffff640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1488104ee0 0x7f1488197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 -- 172.21.15.33:0/4128629286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1488104ee0 msgr2=0x7f1488197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1488104ee0 0x7f1488197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 -- 172.21.15.33:0/4128629286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14880ffe10 msgr2=0x7f14881972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14880ffe10 0x7f14881972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 -- 172.21.15.33:0/4128629286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1470009560 con 0x7f148819bbb0 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f14877fe640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f148000eaa0 tx=0x7f148000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148000cdb0 con 0x7f148819bbb0 2024-01-25T16:21:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14881a3e60 con 0x7f148819bbb0 2024-01-25T16:21:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1480005020 con 0x7f148819bbb0 2024-01-25T16:21:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800116d0 con 0x7f148819bbb0 2024-01-25T16:21:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.944+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14881a4290 con 0x7f148819bbb0 2024-01-25T16:21:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.945+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14881012e0 con 0x7f148819bbb0 2024-01-25T16:21:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.946+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f1480005190 con 0x7f148819bbb0 2024-01-25T16:21:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.946+0000 7f148c8ec640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 0x7f14740a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.947+0000 7f1486ffd640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 0x7f14740a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:43.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.947+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f14800cd1c0 con 0x7f148819bbb0 2024-01-25T16:21:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.947+0000 7f1486ffd640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 0x7f14740a3910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1478009bd0 tx=0x7f1478005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:43.951+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f1480096260 con 0x7f148819bbb0 2024-01-25T16:21:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.170+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1488198640 con 0x7f148819bbb0 2024-01-25T16:21:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.172+0000 7f148c8ec640 1 -- 172.21.15.33:0/4128629286 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f148001f090 con 0x7f148819bbb0 2024-01-25T16:21:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-25T16:21:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 msgr2=0x7f14740a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 0x7f14740a3910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1478009bd0 tx=0x7f1478005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 msgr2=0x7f148819c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f148000eaa0 tx=0x7f148000ef60 comp rx=0 tx=0).stop 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 shutdown_connections 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f14740a1450 0x7f14740a3910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f148819bbb0 0x7f148819c060 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1488104ee0 0x7f1488197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 --2- 172.21.15.33:0/4128629286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14880ffe10 0x7f14881972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 >> 172.21.15.33:0/4128629286 conn(0x7f1488075f70 msgr2=0x7f1488103c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.175+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 shutdown_connections 2024-01-25T16:21:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.176+0000 7f148d8ee640 1 -- 172.21.15.33:0/4128629286 wait complete. 2024-01-25T16:21:44.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-25T16:21:44.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 -- 172.21.15.33:0/2091075046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f410d700 msgr2=0x7fd8f410faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/2091075046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f410d700 0x7fd8f410faf0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd8ec009ca0 tx=0x7fd8ec02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 -- 172.21.15.33:0/2091075046 shutdown_connections 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/2091075046 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f410d700 0x7fd8f410faf0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/2091075046 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f4100fe0 0x7fd8f410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/2091075046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 0x7fd8f4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 -- 172.21.15.33:0/2091075046 >> 172.21.15.33:0/2091075046 conn(0x7fd8f4076c80 msgr2=0x7fd8f4077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 -- 172.21.15.33:0/2091075046 shutdown_connections 2024-01-25T16:21:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.296+0000 7fd8fc52d640 1 -- 172.21.15.33:0/2091075046 wait complete. 2024-01-25T16:21:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 Processor -- start 2024-01-25T16:21:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 -- start start 2024-01-25T16:21:44.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 0x7fd8f4199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f410d700 0x7fd8f419dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f419e300 con 0x7fd8f4100fe0 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f419e470 con 0x7fd8f41006c0 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fc52d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f419e630 con 0x7fd8f410d700 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:44.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60238/0 (socket says 172.21.15.33:60238) 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 -- 172.21.15.33:0/715524367 learned_addr learned my addr 172.21.15.33:0/715524367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8fa2c9640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 0x7fd8f4199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8faaca640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f410d700 0x7fd8f419dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 -- 172.21.15.33:0/715524367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f410d700 msgr2=0x7fd8f419dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f410d700 0x7fd8f419dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 -- 172.21.15.33:0/715524367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 msgr2=0x7fd8f4199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 0x7fd8f4199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:44.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.297+0000 7fd8f9ac8640 1 -- 172.21.15.33:0/715524367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec009900 con 0x7fd8f4100fe0 2024-01-25T16:21:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8f9ac8640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400e9f0 tx=0x7fd8e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4009720 con 0x7fd8f4100fe0 2024-01-25T16:21:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f419e910 con 0x7fd8f4100fe0 2024-01-25T16:21:44.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e400cd80 con 0x7fd8f4100fe0 2024-01-25T16:21:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010430 con 0x7fd8f4100fe0 2024-01-25T16:21:44.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.298+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f41a6190 con 0x7fd8f4100fe0 2024-01-25T16:21:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.299+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f4067250 con 0x7fd8f4100fe0 2024-01-25T16:21:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.300+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7fd8e4005000 con 0x7fd8f4100fe0 2024-01-25T16:21:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.300+0000 7fd8eb7fe640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 0x7fd8e00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:44.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.300+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8799+0+0 (secure 0 0 0) 0x7fd8e4014070 con 0x7fd8f4100fe0 2024-01-25T16:21:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.300+0000 7fd8fa2c9640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 0x7fd8e00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.301+0000 7fd8fa2c9640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 0x7fd8e00a3910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009bd0 tx=0x7fd8f0005f90 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:44.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.304+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7fd8e40965d0 con 0x7fd8f4100fe0 2024-01-25T16:21:44.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a[31413]: 2024-01-25T16:21:44.171+0000 7f8ada4af700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-25T16:21:44.462 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4292464141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/54163334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: osdmap e113: 12 total, 12 up, 12 in 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-25T16:21:44.463 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:44 smithi033 ceph-mon[31417]: pgmap v444: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 704 KiB/s wr, 45 op/s 2024-01-25T16:21:44.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:44.529+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd8f419a790 con 0x7fd8f4100fe0 2024-01-25T16:21:44.628 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.628 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.628 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mds[66602]: -- 172.21.15.152:0/1262787643 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55818718f520 con 0x558187165000 2024-01-25T16:21:44.628 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: starting mds.c at 2024-01-25T16:21:44.629 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.629 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.629 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mds[66298]: -- 172.21.15.152:0/3677743391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b2ad0f3520 con 0x55b2ad0c8c00 2024-01-25T16:21:44.629 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: starting mds.f at 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mds[66922]: -- 172.21.15.152:0/885403225 <== mon.1 v2:172.21.15.152:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d2dcb1f520 con 0x55d2dcaf4c00 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: starting mds.i at 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4292464141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/54163334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: osdmap e113: 12 total, 12 up, 12 in 2024-01-25T16:21:44.631 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-25T16:21:44.632 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:44 smithi152 ceph-mon[36015]: pgmap v444: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 704 KiB/s wr, 45 op/s 2024-01-25T16:21:44.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4292464141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/54163334' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/4128629286' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: osdmap e113: 12 total, 12 up, 12 in 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-01-25T16:21:44.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mon[35642]: pgmap v444: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 704 KiB/s wr, 45 op/s 2024-01-25T16:21:44.665 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.665 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.665 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mds[66275]: -- 172.21.15.76:0/3142817103 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5560016fb520 con 0x5560016d0800 2024-01-25T16:21:44.665 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: starting mds.h at 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mds[65965]: -- 172.21.15.76:0/779977227 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb919eb520 con 0x55eb919c0800 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: starting mds.e at 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: ignoring --setuser ceph since I am not root 2024-01-25T16:21:44.666 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: ignoring --setgroup ceph since I am not root 2024-01-25T16:21:44.667 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-mds[66596]: -- 172.21.15.76:0/1319436676 <== mon.2 v2:172.21.15.76:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565015dd7520 con 0x565015dacc00 2024-01-25T16:21:44.667 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:21:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: starting mds.b at 2024-01-25T16:21:45.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.180+0000 7fd8eb7fe640 1 -- 172.21.15.33:0/715524367 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd8e4099bf0 con 0x7fd8f4100fe0 2024-01-25T16:21:45.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 msgr2=0x7fd8e00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 0x7fd8e00a3910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009bd0 tx=0x7fd8f0005f90 comp rx=0 tx=0).stop 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 msgr2=0x7fd8f4199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400e9f0 tx=0x7fd8e400edf0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 shutdown_connections 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7fd8e00a1450 0x7fd8e00a3910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8f410d700 0x7fd8f419dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8f4100fe0 0x7fd8f4199980 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 --2- 172.21.15.33:0/715524367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8f41006c0 0x7fd8f4199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 >> 172.21.15.33:0/715524367 conn(0x7fd8f4076c80 msgr2=0x7fd8f410ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:45.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 shutdown_connections 2024-01-25T16:21:45.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.182+0000 7fd8fc52d640 1 -- 172.21.15.33:0/715524367 wait complete. 2024-01-25T16:21:45.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-25T16:21:45.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 -- 172.21.15.33:0/1255045122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8514104ea0 msgr2=0x7f851410ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 --2- 172.21.15.33:0/1255045122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8514104ea0 0x7f851410ca70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f850c009cd0 tx=0x7f850c02c0f0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 -- 172.21.15.33:0/1255045122 shutdown_connections 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 --2- 172.21.15.33:0/1255045122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8514104ea0 0x7f851410ca70 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 --2- 172.21.15.33:0/1255045122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514100780 0x7f85141047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 --2- 172.21.15.33:0/1255045122 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f85140ffdd0 0x7f85141001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 -- 172.21.15.33:0/1255045122 >> 172.21.15.33:0/1255045122 conn(0x7f8514076cf0 msgr2=0x7f8514077100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:45.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 -- 172.21.15.33:0/1255045122 shutdown_connections 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 -- 172.21.15.33:0/1255045122 wait complete. 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.304+0000 7f851bb5f640 1 Processor -- start 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 -- start start 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 0x7f8514196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851419a9a0 0x7f851419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851419b390 con 0x7f85140ffdd0 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851419b500 con 0x7f8514104ea0 2024-01-25T16:21:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851bb5f640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851419b670 con 0x7f851419a9a0 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60268/0 (socket says 172.21.15.33:60268) 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 -- 172.21.15.33:0/3101041452 learned_addr learned my addr 172.21.15.33:0/3101041452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f851a0fc640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851419a9a0 0x7f851419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85190fa640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 0x7f8514196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 -- 172.21.15.33:0/3101041452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851419a9a0 msgr2=0x7f851419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851419a9a0 0x7f851419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 -- 172.21.15.33:0/3101041452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 msgr2=0x7f8514196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 0x7f8514196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 -- 172.21.15.33:0/3101041452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009900 con 0x7f85140ffdd0 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85190fa640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 0x7f8514196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.305+0000 7f85198fb640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f851000c970 tx=0x7f851000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.306+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510007bb0 con 0x7f85140ffdd0 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.306+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8510007d10 con 0x7f85140ffdd0 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.306+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85141a2c50 con 0x7f85140ffdd0 2024-01-25T16:21:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.306+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510004e40 con 0x7f85140ffdd0 2024-01-25T16:21:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.306+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85141a30b0 con 0x7f85140ffdd0 2024-01-25T16:21:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.307+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8514067ae0 con 0x7f85140ffdd0 2024-01-25T16:21:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.308+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f8510005000 con 0x7f85140ffdd0 2024-01-25T16:21:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.309+0000 7f850affd640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 0x7f85000a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.309+0000 7f85190fa640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 0x7f85000a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.309+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 8799+0+0 (secure 0 0 0) 0x7f85100cd150 con 0x7f85140ffdd0 2024-01-25T16:21:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.310+0000 7f85190fa640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 0x7f85000a3910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85140ff700 tx=0x7f85040094b0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:45.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.313+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f8510096180 con 0x7f85140ffdd0 2024-01-25T16:21:45.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:45.494+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8514197430 con 0x7f85140ffdd0 2024-01-25T16:21:45.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:21:45.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-25T16:21:45.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-25T16:21:45.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: Cluster is now healthy 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6837/3408992548,v1:172.21.15.152:6839/3408992548] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6834/3727240450,v1:172.21.15.152:6835/3727240450] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6834/3136235515,v1:172.21.15.76:6835/3136235515] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.152:6836/2727226779,v1:172.21.15.152:6838/2727226779] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6836/2863989560,v1:172.21.15.76:6837/2863989560] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: mds.? [v2:172.21.15.76:6838/3189320365,v1:172.21.15.76:6839/3189320365] up:boot 2024-01-25T16:21:45.560 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: fsmap 12 up:standby 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:21:45.561 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:45 smithi033 ceph-mon[31417]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: Cluster is now healthy 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:21:45.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6837/3408992548,v1:172.21.15.152:6839/3408992548] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6834/3727240450,v1:172.21.15.152:6835/3727240450] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6834/3136235515,v1:172.21.15.76:6835/3136235515] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.152:6836/2727226779,v1:172.21.15.152:6838/2727226779] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6836/2863989560,v1:172.21.15.76:6837/2863989560] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: mds.? [v2:172.21.15.76:6838/3189320365,v1:172.21.15.76:6839/3189320365] up:boot 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: fsmap 12 up:standby 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:21:45.629 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:21:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:21:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:21:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:21:45.630 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:45 smithi152 ceph-mon[36015]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:21:45.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-01-25T16:21:45.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: Cluster is now healthy 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/715524367' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6837/3408992548,v1:172.21.15.152:6839/3408992548] up:boot 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6834/3727240450,v1:172.21.15.152:6835/3727240450] up:boot 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6834/3136235515,v1:172.21.15.76:6835/3136235515] up:boot 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.152:6836/2727226779,v1:172.21.15.152:6838/2727226779] up:boot 2024-01-25T16:21:45.664 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6836/2863989560,v1:172.21.15.76:6837/2863989560] up:boot 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: mds.? [v2:172.21.15.76:6838/3189320365,v1:172.21.15.76:6839/3189320365] up:boot 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: fsmap 12 up:standby 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-01-25T16:21:45.665 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:45 smithi076 ceph-mon[35642]: from='mgr.15318 172.21.15.33:0/927067746' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-01-25T16:21:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.298+0000 7f850affd640 1 -- 172.21.15.33:0/3101041452 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v114) v1 ==== 181+0+0 (secure 0 0 0) 0x7f851001d020 con 0x7f85140ffdd0 2024-01-25T16:21:46.297 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-01-25T16:21:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 msgr2=0x7f85000a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 0x7f85000a3910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f85140ff700 tx=0x7f85040094b0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 msgr2=0x7f85141960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f851000c970 tx=0x7f851000ce30 comp rx=0 tx=0).stop 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 shutdown_connections 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f85000a1450 0x7f85000a3910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f851419a9a0 0x7f851419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8514104ea0 0x7f8514196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 --2- 172.21.15.33:0/3101041452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85140ffdd0 0x7f85141960e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.302+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 >> 172.21.15.33:0/3101041452 conn(0x7f8514076cf0 msgr2=0x7f85140fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:46.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.303+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 shutdown_connections 2024-01-25T16:21:46.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.303+0000 7f851bb5f640 1 -- 172.21.15.33:0/3101041452 wait complete. 2024-01-25T16:21:46.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-25T16:21:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 -- 172.21.15.33:0/3093137088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df81018a0 msgr2=0x7f4df81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 --2- 172.21.15.33:0/3093137088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df81018a0 0x7f4df81058e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4dec009930 tx=0x7f4dec02bc30 comp rx=0 tx=0).stop 2024-01-25T16:21:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 -- 172.21.15.33:0/3093137088 shutdown_connections 2024-01-25T16:21:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 --2- 172.21.15.33:0/3093137088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 0x7f4df810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 --2- 172.21.15.33:0/3093137088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df81018a0 0x7f4df81058e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.436+0000 7f4dffbca640 1 --2- 172.21.15.33:0/3093137088 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 0x7f4df81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- 172.21.15.33:0/3093137088 >> 172.21.15.33:0/3093137088 conn(0x7f4df80fc790 msgr2=0x7f4df80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- 172.21.15.33:0/3093137088 shutdown_connections 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- 172.21.15.33:0/3093137088 wait complete. 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 Processor -- start 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- start start 2024-01-25T16:21:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 0x7f4df81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 0x7f4df81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df819c550 con 0x7f4df819bb60 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df819c6c0 con 0x7f4df8105fc0 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.437+0000 7f4dffbca640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df819c830 con 0x7f4df8100ef0 2024-01-25T16:21:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60280/0 (socket says 172.21.15.33:60280) 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 -- 172.21.15.33:0/892480518 learned_addr learned my addr 172.21.15.33:0/892480518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfd966640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 0x7f4df81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfd165640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 0x7f4df81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 -- 172.21.15.33:0/892480518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 msgr2=0x7f4df81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 0x7f4df81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 -- 172.21.15.33:0/892480518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 msgr2=0x7f4df81977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 0x7f4df81977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 -- 172.21.15.33:0/892480518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec009560 con 0x7f4df819bb60 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dfe167640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4de800eb10 tx=0x7f4de800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004b20 con 0x7f4df819bb60 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4de8004c80 con 0x7f4df819bb60 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df81a3e10 con 0x7f4df819bb60 2024-01-25T16:21:46.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8010690 con 0x7f4df819bb60 2024-01-25T16:21:46.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.438+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df81a4240 con 0x7f4df819bb60 2024-01-25T16:21:46.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.439+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df8067ae0 con 0x7f4df819bb60 2024-01-25T16:21:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.440+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 136026+0+0 (secure 0 0 0) 0x7f4de8004fd0 con 0x7f4df819bb60 2024-01-25T16:21:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.441+0000 7f4df6ffd640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 0x7f4de00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-25T16:21:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.441+0000 7f4dfd966640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 0x7f4de00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-25T16:21:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.441+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8303+0+0 (secure 0 0 0) 0x7f4de8014070 con 0x7f4df819bb60 2024-01-25T16:21:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.442+0000 7f4dfd966640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 0x7f4de00a3910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4df00058c0 tx=0x7f4df000a2d0 comp rx=0 tx=0).ready entity=mgr.15318 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-25T16:21:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.445+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+181732 (secure 0 0 0) 0x7f4de8096760 con 0x7f4df819bb60 2024-01-25T16:21:46.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:46 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:46.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:46 smithi033 ceph-mon[31417]: pgmap v445: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 244 KiB/s wr, 12 op/s 2024-01-25T16:21:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:46.624+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4df81985f0 con 0x7f4df819bb60 2024-01-25T16:21:46.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:46 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:46.627 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:46 smithi152 ceph-mon[36015]: pgmap v445: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 244 KiB/s wr, 12 op/s 2024-01-25T16:21:46.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:46 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:46.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:46 smithi076 ceph-mon[35642]: pgmap v445: 97 pgs: 97 active+clean; 120 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 244 KiB/s wr, 12 op/s 2024-01-25T16:21:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.304+0000 7f4df6ffd640 1 -- 172.21.15.33:0/892480518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v115) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4de8099d80 con 0x7f4df819bb60 2024-01-25T16:21:47.303 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 msgr2=0x7f4de00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 0x7f4de00a3910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4df00058c0 tx=0x7f4df000a2d0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 msgr2=0x7f4df819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4de800eb10 tx=0x7f4de800c2d0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 shutdown_connections 2024-01-25T16:21:47.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:6800/2840995268,v1:172.21.15.33:6801/2840995268] conn(0x7f4de00a1450 0x7f4de00a3910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4df819bb60 0x7f4df819c010 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8105fc0 0x7f4df81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 --2- 172.21.15.33:0/892480518 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4df8100ef0 0x7f4df81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 >> 172.21.15.33:0/892480518 conn(0x7f4df80fc790 msgr2=0x7f4df80fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 shutdown_connections 2024-01-25T16:21:47.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-01-25T16:21:47.308+0000 7f4dffbca640 1 -- 172.21.15.33:0/892480518 wait complete. 2024-01-25T16:21:47.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:21:47.360 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:21:47.416 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:21:47.452 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-01-25T16:21:47.453 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-25T16:21:47.467 DEBUG:teuthology.orchestra.run.smithi076:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-25T16:21:47.482 DEBUG:teuthology.orchestra.run.smithi152:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-01-25T16:21:47.510 INFO:tasks.cephadm:Stopping all daemons... 2024-01-25T16:21:47.510 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-01-25T16:21:47.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a 2024-01-25T16:21:47.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-25T16:21:47.558 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 ceph-mon[31417]: osdmap e114: 12 total, 12 up, 12 in 2024-01-25T16:21:47.559 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 ceph-mon[31417]: from='client.? 172.21.15.33:0/892480518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:47.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-25T16:21:47.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:47 smithi152 ceph-mon[36015]: osdmap e114: 12 total, 12 up, 12 in 2024-01-25T16:21:47.628 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:47 smithi152 ceph-mon[36015]: from='client.? 172.21.15.33:0/892480518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:47.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/3101041452' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-01-25T16:21:47.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:47 smithi076 ceph-mon[35642]: osdmap e114: 12 total, 12 up, 12 in 2024-01-25T16:21:47.663 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:47 smithi076 ceph-mon[35642]: from='client.? 172.21.15.33:0/892480518' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-01-25T16:21:47.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 systemd[1]: Stopping Ceph mon.a for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:48.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a[31413]: 2024-01-25T16:21:47.812+0000 7f8adfcba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:21:48.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a[31413]: 2024-01-25T16:21:47.812+0000 7f8adfcba700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-01-25T16:21:48.309 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:48 smithi033 podman[92364]: 2024-01-25 16:21:48.018291172 +0000 UTC m=+0.313763068 container died 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, org.label-schema.build-date=20240123, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-01-25T16:21:48.636 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:48 smithi033 podman[92364]: 2024-01-25 16:21:48.3889527 +0000 UTC m=+0.684424598 container cleanup 6e997038abfe22bb7e1d680d1a3d02204beeb19c060139bd55a30cf4a021cc17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:21:48.637 INFO:journalctl@ceph.mon.a.smithi033.stdout:Jan 25 16:21:48 smithi033 bash[92364]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-a 2024-01-25T16:21:48.810 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.a.service' 2024-01-25T16:21:48.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:48.862 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-01-25T16:21:48.862 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-01-25T16:21:48.863 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.b 2024-01-25T16:21:49.163 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:48 smithi076 systemd[1]: Stopping Ceph mon.b for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:49.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-b[35637]: 2024-01-25T16:21:49.218+0000 7f452d878700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:21:49.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-b[35637]: 2024-01-25T16:21:49.218+0000 7f452d878700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-01-25T16:21:49.596 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:49 smithi076 podman[72935]: 2024-01-25 16:21:49.300262152 +0000 UTC m=+0.225727040 container died 61742daf75f1ee7e1ef0f2cc873077855997d096b0cf267484a3da942ed67139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, io.buildah.version=1.31.3) 2024-01-25T16:21:49.913 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:49 smithi076 podman[72935]: 2024-01-25 16:21:49.690427332 +0000 UTC m=+0.615892194 container cleanup 61742daf75f1ee7e1ef0f2cc873077855997d096b0cf267484a3da942ed67139 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:21:49.914 INFO:journalctl@ceph.mon.b.smithi076.stdout:Jan 25 16:21:49 smithi076 bash[72935]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-b 2024-01-25T16:21:50.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.b.service' 2024-01-25T16:21:50.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:50.161 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-01-25T16:21:50.161 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-01-25T16:21:50.162 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.c 2024-01-25T16:21:50.473 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:50 smithi152 systemd[1]: Stopping Ceph mon.c for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:50.877 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:50 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-c[36000]: 2024-01-25T16:21:50.475+0000 7f1501d82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:21:50.878 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:50 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-c[36000]: 2024-01-25T16:21:50.475+0000 7f1501d82700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-01-25T16:21:50.878 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:50 smithi152 podman[73508]: 2024-01-25 16:21:50.690244626 +0000 UTC m=+0.329371595 container died 3d368563f5e33e01f1236aee5ad9d0fce52093df8c43a35a0f71d5f3e154272c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-c, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240123, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-01-25T16:21:51.325 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:51 smithi152 podman[73508]: 2024-01-25 16:21:51.043039539 +0000 UTC m=+0.682166489 container cleanup 3d368563f5e33e01f1236aee5ad9d0fce52093df8c43a35a0f71d5f3e154272c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-c, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux ) 2024-01-25T16:21:51.325 INFO:journalctl@ceph.mon.c.smithi152.stdout:Jan 25 16:21:51 smithi152 bash[73508]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mon-c 2024-01-25T16:21:51.462 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mon.c.service' 2024-01-25T16:21:51.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:51.516 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-01-25T16:21:51.516 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-01-25T16:21:51.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x 2024-01-25T16:21:51.809 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:21:51 smithi033 systemd[1]: Stopping Ceph mgr.x for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:52.124 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:21:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x[31646]: 2024-01-25T16:21:51.825+0000 7efd88850700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:21:52.125 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:21:51 smithi033 podman[92485]: 2024-01-25 16:21:51.857098065 +0000 UTC m=+0.136517963 container died ea9ea84c77890c80f2551c4e9e8e53bb6c60d5e9fbff031080425527201701a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123) 2024-01-25T16:21:52.512 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:21:52 smithi033 podman[92485]: 2024-01-25 16:21:52.243917176 +0000 UTC m=+0.523337067 container cleanup ea9ea84c77890c80f2551c4e9e8e53bb6c60d5e9fbff031080425527201701a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:661946e7044723130c03627323f6a879a9848cb5, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-01-25T16:21:52.512 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Jan 25 16:21:52 smithi033 bash[92485]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-x 2024-01-25T16:21:52.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.x.service' 2024-01-25T16:21:52.673 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:52.673 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-01-25T16:21:52.673 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-01-25T16:21:52.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.y 2024-01-25T16:21:52.988 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:21:52 smithi076 systemd[1]: Stopping Ceph mgr.y for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:53.293 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:21:52 smithi076 podman[73059]: 2024-01-25 16:21:52.991191047 +0000 UTC m=+0.133222655 container died 015b1b48e706de252ec6dee60ce03781f8522089800f98c73857fe6d063d988e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-01-25T16:21:53.644 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:21:53 smithi076 podman[73059]: 2024-01-25 16:21:53.387649779 +0000 UTC m=+0.529681389 container cleanup 015b1b48e706de252ec6dee60ce03781f8522089800f98c73857fe6d063d988e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240123, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=) 2024-01-25T16:21:53.645 INFO:journalctl@ceph.mgr.y.smithi076.stdout:Jan 25 16:21:53 smithi076 bash[73059]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-y 2024-01-25T16:21:53.805 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.y.service' 2024-01-25T16:21:53.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:53.843 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-01-25T16:21:53.843 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-01-25T16:21:53.843 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.z 2024-01-25T16:21:54.127 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:53 smithi152 systemd[1]: Stopping Ceph mgr.z for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:54.513 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:54 smithi152 podman[73633]: 2024-01-25 16:21:54.177202369 +0000 UTC m=+0.141300398 container died 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123) 2024-01-25T16:21:54.877 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:54 smithi152 podman[73633]: 2024-01-25 16:21:54.514753444 +0000 UTC m=+0.478851473 container cleanup 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:21:54.878 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:54 smithi152 bash[73633]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z 2024-01-25T16:21:54.878 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:54 smithi152 podman[73647]: 2024-01-25 16:21:54.691531123 +0000 UTC m=+0.506345888 container remove 249c52d1598dcc1224200802caf6128227be3ef524fd66d7295111eeacab1704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, ceph=True) 2024-01-25T16:21:54.878 INFO:journalctl@ceph.mgr.z.smithi152.stdout:Jan 25 16:21:54 smithi152 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-01-25T16:21:54.921 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mgr.z.service' 2024-01-25T16:21:54.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:21:54.958 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-01-25T16:21:54.958 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-01-25T16:21:54.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.0 2024-01-25T16:21:55.244 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:21:55 smithi033 systemd[1]: Stopping Ceph osd.0 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:21:55.559 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:21:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0[46056]: 2024-01-25T16:21:55.247+0000 7f22a6c73700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:21:55.559 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:21:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0[46056]: 2024-01-25T16:21:55.247+0000 7f22a6c73700 -1 osd.0 115 *** Got signal Terminated *** 2024-01-25T16:21:55.559 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:21:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0[46056]: 2024-01-25T16:21:55.247+0000 7f22a6c73700 -1 osd.0 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:01.087 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:00 smithi033 podman[92606]: 2024-01-25 16:22:00.812846603 +0000 UTC m=+5.658052772 container died d36bd42a6be6390f10cf398d6f16dbe1a9d050aec3787937e1b4b20b58dbfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-01-25T16:22:01.437 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:01 smithi033 podman[92606]: 2024-01-25 16:22:01.181403746 +0000 UTC m=+6.026609915 container cleanup d36bd42a6be6390f10cf398d6f16dbe1a9d050aec3787937e1b4b20b58dbfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-25T16:22:01.438 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:01 smithi033 bash[92606]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0 2024-01-25T16:22:01.785 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:01 smithi033 podman[92624]: 2024-01-25 16:22:01.440947552 +0000 UTC m=+0.624883503 container remove d36bd42a6be6390f10cf398d6f16dbe1a9d050aec3787937e1b4b20b58dbfeec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123) 2024-01-25T16:22:02.043 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:01 smithi033 podman[92713]: 2024-01-25 16:22:01.693037576 +0000 UTC m=+0.018088361 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:02.043 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:01 smithi033 podman[92713]: 2024-01-25 16:22:01.791797937 +0000 UTC m=+0.116848762 container create 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-01-25T16:22:02.043 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:02 smithi033 podman[92713]: 2024-01-25 16:22:02.046516711 +0000 UTC m=+0.371567504 container init 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-01-25T16:22:02.309 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:02 smithi033 podman[92713]: 2024-01-25 16:22:02.050066112 +0000 UTC m=+0.375116893 container start 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-01-25T16:22:02.309 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:02 smithi033 podman[92713]: 2024-01-25 16:22:02.1297618 +0000 UTC m=+0.454812581 container attach 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123) 2024-01-25T16:22:02.612 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:02 smithi033 podman[92713]: 2024-01-25 16:22:02.331228939 +0000 UTC m=+0.656279730 container died 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD) 2024-01-25T16:22:02.876 INFO:journalctl@ceph.osd.0.smithi033.stdout:Jan 25 16:22:02 smithi033 podman[92740]: 2024-01-25 16:22:02.857913055 +0000 UTC m=+0.517267867 container remove 2f384f27204720be7a055e3d291ea0610ee13373f814a58d4c068bec21c65d90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:22:02.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.0.service' 2024-01-25T16:22:02.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:02.923 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-01-25T16:22:02.923 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-01-25T16:22:02.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.1 2024-01-25T16:22:03.230 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:02 smithi076 systemd[1]: Stopping Ceph osd.1 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:03.556 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1[40254]: 2024-01-25T16:22:03.232+0000 7f56060aa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:03.557 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1[40254]: 2024-01-25T16:22:03.232+0000 7f56060aa700 -1 osd.1 115 *** Got signal Terminated *** 2024-01-25T16:22:03.557 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1[40254]: 2024-01-25T16:22:03.232+0000 7f56060aa700 -1 osd.1 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:09.006 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:08 smithi076 podman[73181]: 2024-01-25 16:22:08.73952112 +0000 UTC m=+5.619198280 container died 629705f7c28f2d5c6b537aa5623b0270d5f08a5a86536ea2a0e29c0ac1373895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1, GIT_CLEAN=True, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-25T16:22:09.330 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73181]: 2024-01-25 16:22:09.099293922 +0000 UTC m=+5.978971063 container cleanup 629705f7c28f2d5c6b537aa5623b0270d5f08a5a86536ea2a0e29c0ac1373895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:09.330 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 bash[73181]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1 2024-01-25T16:22:09.624 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73210]: 2024-01-25 16:22:09.333671892 +0000 UTC m=+0.593397144 container remove 629705f7c28f2d5c6b537aa5623b0270d5f08a5a86536ea2a0e29c0ac1373895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.31.3) 2024-01-25T16:22:09.913 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73300]: 2024-01-25 16:22:09.531261045 +0000 UTC m=+0.017753997 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:09.913 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73300]: 2024-01-25 16:22:09.643300734 +0000 UTC m=+0.129793674 container create 926ed6c8f294e42e3125ba8b25cbe0121e850dfe61e281a9055ac96ba862330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, GIT_BRANCH=HEAD) 2024-01-25T16:22:09.913 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73300]: 2024-01-25 16:22:09.853933271 +0000 UTC m=+0.340426211 container init 926ed6c8f294e42e3125ba8b25cbe0121e850dfe61e281a9055ac96ba862330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True) 2024-01-25T16:22:09.914 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73300]: 2024-01-25 16:22:09.857022343 +0000 UTC m=+0.343515284 container start 926ed6c8f294e42e3125ba8b25cbe0121e850dfe61e281a9055ac96ba862330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:10.222 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:09 smithi076 podman[73300]: 2024-01-25 16:22:09.953677362 +0000 UTC m=+0.440170305 container attach 926ed6c8f294e42e3125ba8b25cbe0121e850dfe61e281a9055ac96ba862330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-01-25T16:22:10.663 INFO:journalctl@ceph.osd.1.smithi076.stdout:Jan 25 16:22:10 smithi076 podman[73300]: 2024-01-25 16:22:10.226049156 +0000 UTC m=+0.712542088 container died 926ed6c8f294e42e3125ba8b25cbe0121e850dfe61e281a9055ac96ba862330e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-01-25T16:22:10.973 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.1.service' 2024-01-25T16:22:11.018 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:11.018 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-01-25T16:22:11.018 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-01-25T16:22:11.019 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.2 2024-01-25T16:22:11.316 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:11 smithi152 systemd[1]: Stopping Ceph osd.2 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:11.579 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:22:11.319+0000 7f560fce5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:11.579 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:22:11.319+0000 7f560fce5700 -1 osd.2 115 *** Got signal Terminated *** 2024-01-25T16:22:11.579 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2[41189]: 2024-01-25T16:22:11.319+0000 7f560fce5700 -1 osd.2 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:16.958 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:16 smithi152 podman[73754]: 2024-01-25 16:22:16.682582853 +0000 UTC m=+5.467283294 container died 89688a3f56e39bf98f37833a6d8a211868db64903f2d61ce5ba6840c3d14d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:17.308 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73754]: 2024-01-25 16:22:17.052255795 +0000 UTC m=+5.836956234 container cleanup 89688a3f56e39bf98f37833a6d8a211868db64903f2d61ce5ba6840c3d14d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-25T16:22:17.309 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 bash[73754]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2 2024-01-25T16:22:17.595 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73785]: 2024-01-25 16:22:17.312045229 +0000 UTC m=+0.627950750 container remove 89688a3f56e39bf98f37833a6d8a211868db64903f2d61ce5ba6840c3d14d074 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-01-25T16:22:17.845 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73873]: 2024-01-25 16:22:17.502299041 +0000 UTC m=+0.017304790 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:17.846 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73873]: 2024-01-25 16:22:17.629995705 +0000 UTC m=+0.145001484 container create 6d54dbafd0f1ee2a48887ffb169419a845459f0a30c298c076742961636f927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2) 2024-01-25T16:22:18.127 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73873]: 2024-01-25 16:22:17.849488673 +0000 UTC m=+0.364494408 container init 6d54dbafd0f1ee2a48887ffb169419a845459f0a30c298c076742961636f927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123) 2024-01-25T16:22:18.127 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73873]: 2024-01-25 16:22:17.852725056 +0000 UTC m=+0.367730795 container start 6d54dbafd0f1ee2a48887ffb169419a845459f0a30c298c076742961636f927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_CLEAN=True) 2024-01-25T16:22:18.128 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:17 smithi152 podman[73873]: 2024-01-25 16:22:17.941017704 +0000 UTC m=+0.456023456 container attach 6d54dbafd0f1ee2a48887ffb169419a845459f0a30c298c076742961636f927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, GIT_CLEAN=True) 2024-01-25T16:22:18.432 INFO:journalctl@ceph.osd.2.smithi152.stdout:Jan 25 16:22:18 smithi152 podman[73873]: 2024-01-25 16:22:18.131761239 +0000 UTC m=+0.646766977 container died 6d54dbafd0f1ee2a48887ffb169419a845459f0a30c298c076742961636f927d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-01-25T16:22:18.704 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.2.service' 2024-01-25T16:22:18.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:18.748 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-01-25T16:22:18.748 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-01-25T16:22:18.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.3 2024-01-25T16:22:19.059 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:18 smithi033 systemd[1]: Stopping Ceph osd.3 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:19.059 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:22:19.025+0000 7f155cefe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:19.059 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:22:19.025+0000 7f155cefe700 -1 osd.3 115 *** Got signal Terminated *** 2024-01-25T16:22:19.059 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:19 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:22:19.025+0000 7f155cefe700 -1 osd.3 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:21.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:20 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:20.869+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:22.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:21.852+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:22.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:21 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:21.669+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:22.308 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:21 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:21.873+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:23.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:22 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:22.696+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:23.179 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:22 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:22.859+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:23.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:22.884+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:23.524 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:23 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3[52799]: 2024-01-25T16:22:23.182+0000 7f1555cf2700 -1 osd.3 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:56.984480+0000 front 2024-01-25T16:21:56.984448+0000 (oldest deadline 2024-01-25T16:22:22.284592+0000) 2024-01-25T16:22:23.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:23 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:23.527+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:24.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:23 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:23.678+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:24.193 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:23.880+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:24.309 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:23 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:23.837+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:24.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:24.196+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:24.658 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:24 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:24.571+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:24.658 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:24 smithi033 podman[92820]: 2024-01-25 16:22:24.400776809 +0000 UTC m=+5.452599862 container died a2dade74195235f29e18a328a7a7a1568b35851649612a3489d94b8f00f3fc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123) 2024-01-25T16:22:25.040 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:24 smithi033 podman[92820]: 2024-01-25 16:22:24.750929739 +0000 UTC m=+5.802752786 container cleanup a2dade74195235f29e18a328a7a7a1568b35851649612a3489d94b8f00f3fc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:25.040 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:24 smithi033 bash[92820]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3 2024-01-25T16:22:25.040 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:24 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:24.853+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:25.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:24 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:24.668+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:25.224 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:24.927+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:25.308 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92849]: 2024-01-25 16:22:25.043636662 +0000 UTC m=+0.639203755 container remove a2dade74195235f29e18a328a7a7a1568b35851649612a3489d94b8f00f3fc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:22:25.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:25.225+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:25.676 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:25 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:25.550+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:25.676 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.267476532 +0000 UTC m=+0.017502561 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:25.676 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.369452548 +0000 UTC m=+0.119478612 container create 1755fac1e106b4ce29936fb7d15e8337a13bcff11153427bfe9670ffb2587ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3-deactivate, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:22:25.677 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.596705947 +0000 UTC m=+0.346731977 container init 1755fac1e106b4ce29936fb7d15e8337a13bcff11153427bfe9670ffb2587ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-01-25T16:22:25.677 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.599990446 +0000 UTC m=+0.350016474 container start 1755fac1e106b4ce29936fb7d15e8337a13bcff11153427bfe9670ffb2587ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-01-25T16:22:25.677 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.679860704 +0000 UTC m=+0.429886747 container attach 1755fac1e106b4ce29936fb7d15e8337a13bcff11153427bfe9670ffb2587ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:26.058 INFO:journalctl@ceph.osd.3.smithi033.stdout:Jan 25 16:22:25 smithi033 podman[92938]: 2024-01-25 16:22:25.887237672 +0000 UTC m=+0.637263695 container died 1755fac1e106b4ce29936fb7d15e8337a13bcff11153427bfe9670ffb2587ac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_CLEAN=True) 2024-01-25T16:22:26.059 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:25 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:25.807+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:26.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:25.877+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:26.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:25 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:25.670+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:26.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.3.service' 2024-01-25T16:22:26.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:26.195+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:26.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:26.629 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-01-25T16:22:26.630 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-01-25T16:22:26.630 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.4 2024-01-25T16:22:26.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:26 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:26.711+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:26.913 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:26 smithi076 systemd[1]: Stopping Ceph osd.4 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:26.914 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:26 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:22:26.909+0000 7f7286fe9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:26.914 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:26 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:22:26.909+0000 7f7286fe9700 -1 osd.4 115 *** Got signal Terminated *** 2024-01-25T16:22:26.914 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:26 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:22:26.909+0000 7f7286fe9700 -1 osd.4 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:27.059 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:26.804+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:27.059 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:26 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:26.573+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:27.185 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:26.924+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:27.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:27 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:27.188+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:27.796 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:27 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:27.549+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:28.058 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:27 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:27.799+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:28.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:27 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:27.715+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:28.195 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:28 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:28.149+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:28.196 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:27 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:27.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:29.058 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:28 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:28.757+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:29.059 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:28 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:28.587+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:29.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:28 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:28.666+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:29.377 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:29 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:29.167+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:29.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:28 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:28.930+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:29.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:29 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:29.635+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:29.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:29 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:29.799+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:30.058 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:29 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:29.718+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:30.059 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:29 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:29.611+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:30.190 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:29 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:29.898+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:30.626 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:30 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:22:30.234+0000 7f7283de5700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:04.611263+0000 front 2024-01-25T16:22:04.611294+0000 (oldest deadline 2024-01-25T16:22:29.911402+0000) 2024-01-25T16:22:30.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:30 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:30.193+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:30.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:30 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:30.193+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:30.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:30 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:30.786+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:30.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:30 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:30.628+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:31.058 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:30 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:30.576+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:31.059 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:30 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:30.703+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:31.059 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:30 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:30.703+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:31.192 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:30 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:30.897+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:31.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:31 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:31.195+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:31.628 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:31 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:31.195+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:31.637 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:31 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4[47120]: 2024-01-25T16:22:31.251+0000 7f7283de5700 -1 osd.4 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:04.611263+0000 front 2024-01-25T16:22:04.611294+0000 (oldest deadline 2024-01-25T16:22:29.911402+0000) 2024-01-25T16:22:31.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:31 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:31.669+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:31.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:31 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:31.669+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:31.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:31 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:31.529+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:31.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:31 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:31.529+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:31.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:31 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:31.782+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:31.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:31 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:31.640+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:32.235 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:31 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:31.936+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:32.590 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:32 smithi076 podman[73406]: 2024-01-25 16:22:32.306178007 +0000 UTC m=+5.477104725 container died e99e859568f322b9ed8b5ebf13461213ce9594790adaab70439c4f635e69f341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux ) 2024-01-25T16:22:32.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:32 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:32.238+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:32.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:32 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:32.238+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:32.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:32 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:32.648+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:32.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:32 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:32.648+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:32.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:32 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:32.545+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:32.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:32 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:32.545+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:32.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:32 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:32.768+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:32.913 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:32 smithi076 podman[73406]: 2024-01-25 16:22:32.69236008 +0000 UTC m=+5.863286804 container cleanup e99e859568f322b9ed8b5ebf13461213ce9594790adaab70439c4f635e69f341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:22:32.913 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:32 smithi076 bash[73406]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4 2024-01-25T16:22:32.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:32 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:32.622+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:33.263 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:32 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:32.947+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:33.264 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:32 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:32.975+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:33.264 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:32 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:32.975+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:33.298 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:32 smithi076 podman[73425]: 2024-01-25 16:22:32.960162043 +0000 UTC m=+0.651330391 container remove e99e859568f322b9ed8b5ebf13461213ce9594790adaab70439c4f635e69f341 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240123) 2024-01-25T16:22:33.592 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.302542751 +0000 UTC m=+0.112230512 container create c26300d37bf7cf7c4e415af94b5c65cc888a76906fac76c45637d6f28069fee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=) 2024-01-25T16:22:33.592 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.20801116 +0000 UTC m=+0.017698935 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:33.592 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.529998028 +0000 UTC m=+0.339685799 container init c26300d37bf7cf7c4e415af94b5c65cc888a76906fac76c45637d6f28069fee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:22:33.592 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.533094942 +0000 UTC m=+0.342782712 container start c26300d37bf7cf7c4e415af94b5c65cc888a76906fac76c45637d6f28069fee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:33.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:33.266+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:33.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:33.266+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:33.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:33 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:33.544+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:33.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:33 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:33.544+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:33.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:33 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:33.609+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:33.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:33 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:33.609+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:33.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:33 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:33.771+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:33.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:33 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:33.594+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:33.913 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.617334581 +0000 UTC m=+0.427022359 container attach c26300d37bf7cf7c4e415af94b5c65cc888a76906fac76c45637d6f28069fee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:33.914 INFO:journalctl@ceph.osd.4.smithi076.stdout:Jan 25 16:22:33 smithi076 podman[73518]: 2024-01-25 16:22:33.812083348 +0000 UTC m=+0.621771120 container died c26300d37bf7cf7c4e415af94b5c65cc888a76906fac76c45637d6f28069fee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3) 2024-01-25T16:22:34.299 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:33.916+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:34.300 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:33.991+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:34.300 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:33 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:33.991+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:34.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.4.service' 2024-01-25T16:22:34.421 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:34.422 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-01-25T16:22:34.422 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-01-25T16:22:34.422 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.5 2024-01-25T16:22:34.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:34.302+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:34.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:34.302+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:34.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 systemd[1]: Stopping Ceph osd.5 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:34.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:34 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:34.530+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:34.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:34 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:34.530+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:34.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:34 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:34.576+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:34.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:34 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:34.576+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:34.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:34 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:34.726+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:34.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:34 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:34.639+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:35.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:34.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:35.127 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:34.756+0000 7fecdf4b0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:35.128 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:34.756+0000 7fecdf4b0700 -1 osd.5 115 *** Got signal Terminated *** 2024-01-25T16:22:35.128 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:34.756+0000 7fecdf4b0700 -1 osd.5 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:35.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:34.978+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:35.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:34 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:34.978+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:35.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:35 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:35.260+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:35.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:35 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:35.260+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:35.810 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:35.501+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:35.811 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:35.501+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:35.811 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:35.561+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:35.811 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:35.561+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:35.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:35 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:35.742+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:35.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:35 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:35.611+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:36.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:35 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:35.877+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:36.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:35 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:35.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:36.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:35 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:35.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:36.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:36.274+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:36.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:36.274+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:36.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:36 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:36.481+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:36.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:36 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:36.481+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:36.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:36 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:36.579+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:36.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:36 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:36.579+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:36.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:36 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:36.640+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:36.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:36 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:36.734+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:37.312 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:36.913+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:37.312 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:36.987+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:37.312 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:36 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:36.987+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:37.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:37.314+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:37.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:37.314+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:37.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:37.570+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:37.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:37.570+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:37.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:37.445+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:37.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:37.445+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:37.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:37 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:37.445+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:37.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:37 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:37.652+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:37.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:37 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:37.703+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:37.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:37 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:37.703+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:38.316 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:37.898+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:38.316 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:37.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:38.316 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:37.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:38.317 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:37 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:37.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:38.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:38.318+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:38.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:38.318+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:38.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:38 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:38.549+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:38.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:38 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:38.549+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:38.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:38 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:38.440+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:38.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:38 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:38.440+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:38.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:38 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:38.440+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:38.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:38 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:38.611+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:38.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:38 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:38.611+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:38.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:38 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:38.748+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:38.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:38 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:38.748+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:39.347 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:38.938+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:39.347 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:38.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:39.347 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:38.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:39.348 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:38 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:38.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:39.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:39.350+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.691774+0000 front 2024-01-25T16:21:58.691708+0000 (oldest deadline 2024-01-25T16:22:23.991795+0000) 2024-01-25T16:22:39.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:39.350+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:06.694364+0000 front 2024-01-25T16:22:06.694268+0000 (oldest deadline 2024-01-25T16:22:29.594326+0000) 2024-01-25T16:22:39.627 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5[47878]: 2024-01-25T16:22:39.350+0000 7fecd82a4700 -1 osd.5 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.295467+0000 front 2024-01-25T16:22:14.295493+0000 (oldest deadline 2024-01-25T16:22:38.995701+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:39.424+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:39.424+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:39.424+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:39.594+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:39.594+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:39.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:39 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:39.594+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:39.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:39 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:39.797+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:39.913 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:39 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:39.797+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:39.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:39 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:39.571+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:39.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:39 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:39.571+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:40.288 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:39.907+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:40.289 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:39.902+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:40.289 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:39.902+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:40.289 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:39 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:39.902+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:40.556 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:40 smithi152 podman[73979]: 2024-01-25 16:22:40.292545392 +0000 UTC m=+5.673389637 container died 069d1930cdd0d01494404e676324f0627fd948d0a45eaac8036879eb1899b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240123) 2024-01-25T16:22:40.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:40.469+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:40.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:40.469+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:40.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:40.469+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:40.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:40.557+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:40.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:40.557+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:40.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:40.557+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:40.843 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:40 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:40.529+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:40.843 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:40 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:40.529+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:40.877 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:40 smithi152 podman[73979]: 2024-01-25 16:22:40.640511804 +0000 UTC m=+6.021356049 container cleanup 069d1930cdd0d01494404e676324f0627fd948d0a45eaac8036879eb1899b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, ceph=True, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-01-25T16:22:40.877 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:40 smithi152 bash[73979]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5 2024-01-25T16:22:41.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:40 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:40.845+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:41.229 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:40 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:40.845+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:41.377 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[73997]: 2024-01-25 16:22:41.067159918 +0000 UTC m=+0.772978026 container remove 069d1930cdd0d01494404e676324f0627fd948d0a45eaac8036879eb1899b317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-01-25T16:22:41.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:40.883+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:41.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:40.883+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:41.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:40.883+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:41.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:40.883+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:41.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:40 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:40.883+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:41.658 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.301189036 +0000 UTC m=+0.017746399 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:41.658 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.417965138 +0000 UTC m=+0.134522498 container create 4fec71125e6802d7543304949cf3bad81f9a5640232e130e18495a22a15f3a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3) 2024-01-25T16:22:41.795 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:41 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:41.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:41.795 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:41 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:41.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:41.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:41.515+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:41.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:41.515+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:41.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:41.515+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:41.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:41.509+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:41.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:41.509+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:41.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:41 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:41.509+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:41.926 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.662486047 +0000 UTC m=+0.379043409 container init 4fec71125e6802d7543304949cf3bad81f9a5640232e130e18495a22a15f3a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-01-25T16:22:41.926 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.665614163 +0000 UTC m=+0.382171528 container start 4fec71125e6802d7543304949cf3bad81f9a5640232e130e18495a22a15f3a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:41.927 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.795781371 +0000 UTC m=+0.512338743 container attach 4fec71125e6802d7543304949cf3bad81f9a5640232e130e18495a22a15f3a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123) 2024-01-25T16:22:42.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:41 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:41.797+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:42.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:41 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:41.797+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:42.228 INFO:journalctl@ceph.osd.5.smithi152.stdout:Jan 25 16:22:41 smithi152 podman[74088]: 2024-01-25 16:22:41.942045375 +0000 UTC m=+0.658602738 container died 4fec71125e6802d7543304949cf3bad81f9a5640232e130e18495a22a15f3a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:42.229 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:41.933+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:42.229 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:41.933+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:42.229 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:41.929+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:42.229 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:41.929+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:42.229 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:41 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:41.929+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:42.519 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.5.service' 2024-01-25T16:22:42.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:42.562 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-01-25T16:22:42.562 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-01-25T16:22:42.562 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.6 2024-01-25T16:22:42.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:42.472+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:42.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:42.472+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:42.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:42.472+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:42.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.484+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:42.810 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.484+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:42.810 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.484+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:42.810 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 systemd[1]: Stopping Ceph osd.6 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:42.841 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:42 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:42.507+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:42.841 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:42 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:42.507+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:43.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:42 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:42.911+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:43.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:42 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:42.911+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:43.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:42 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:42.923+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:43.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:42 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:42.923+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:43.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:42 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:42.923+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:43.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:42 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:42.843+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:43.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:42 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:42.843+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:43.308 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.868+0000 7f8eb7f71700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:43.309 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.868+0000 7f8eb7f71700 -1 osd.6 115 *** Got signal Terminated *** 2024-01-25T16:22:43.309 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:42 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:42.868+0000 7f8eb7f71700 -1 osd.6 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:43.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:43.481+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:43.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:43.481+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:43.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:43.481+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:43.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:43.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:43.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:43.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:43.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:43 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:43.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:43.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:43.493+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:43.885 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:43.493+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:44.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:43.867+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:44.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:43.867+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:44.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:43.919+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:44.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:43.919+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:44.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:43 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:43.919+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:44.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:43.887+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:44.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:43 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:43.887+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:44.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:44.437+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:44.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:44.437+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:44.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:44.437+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:44.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:44.529+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:44.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:44.529+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:44.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:44 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:44.529+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:44.841 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:44.456+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:44.841 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:44.456+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:45.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:44.819+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:45.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:44.819+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:45.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:44.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:45.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:44.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:45.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:44 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:44.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:45.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:44.844+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:45.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:44 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:44.844+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:45.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:45.420+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:45.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:45.420+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:45.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:45.420+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:45.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:45.499+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:45.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:45.499+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:45.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:45.499+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:45.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:45.447+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:45.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:45.447+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:45.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:45.447+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:46.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:45.831+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:46.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:45.831+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:46.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:45.904+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:46.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:45.904+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:46.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:45 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:45.904+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:46.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:45.887+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:46.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:45 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:45.887+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:46.662 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:46.411+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:46.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:46.411+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:46.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:46.411+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:46.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:46.515+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:46.515+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:46.515+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:46.515+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:46.387+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:46.387+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:46.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:46.387+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:46.810 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:46 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:46.387+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.596564+0000 front 2024-01-25T16:22:23.596585+0000 (oldest deadline 2024-01-25T16:22:45.896676+0000) 2024-01-25T16:22:47.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:46.818+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:47.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:46.818+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:47.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:46.940+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:47.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:46.940+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:47.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:46.940+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:47.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:46 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:46.940+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:47.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:46.936+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:47.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:46.936+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:47.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:46 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:46.936+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:47.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:47.360+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:58.591367+0000 front 2024-01-25T16:21:58.591312+0000 (oldest deadline 2024-01-25T16:22:23.291356+0000) 2024-01-25T16:22:47.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:47.360+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.992724+0000 front 2024-01-25T16:22:07.992787+0000 (oldest deadline 2024-01-25T16:22:30.892779+0000) 2024-01-25T16:22:47.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:47.360+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:13.194287+0000 front 2024-01-25T16:22:13.194214+0000 (oldest deadline 2024-01-25T16:22:36.694211+0000) 2024-01-25T16:22:47.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6[59551]: 2024-01-25T16:22:47.360+0000 7f8eb4d6d700 -1 osd.6 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.596564+0000 front 2024-01-25T16:22:23.596585+0000 (oldest deadline 2024-01-25T16:22:45.896676+0000) 2024-01-25T16:22:47.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:47.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:47.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:47.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:47.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:47.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:47.810 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:47 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:47.496+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:47.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:47.430+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:47.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:47.430+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:47.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:47.430+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:48.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:47.815+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:48.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:47.815+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:48.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:47.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:48.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:47.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:48.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:47.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:48.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:47 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:47.934+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:48.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:47.928+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:48.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:47.928+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:48.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:47 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:47.928+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:48.506 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:48 smithi033 podman[93044]: 2024-01-25 16:22:48.253358665 +0000 UTC m=+5.479743130 container died 7c191be84bcd2347d7e87b7ed37fa72a650ee52d18f8e34a92ac99d8bc12c18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123) 2024-01-25T16:22:48.808 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:48 smithi033 podman[93044]: 2024-01-25 16:22:48.726295466 +0000 UTC m=+5.952679926 container cleanup 7c191be84bcd2347d7e87b7ed37fa72a650ee52d18f8e34a92ac99d8bc12c18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:48.809 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:48 smithi033 bash[93044]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6 2024-01-25T16:22:48.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:48.517+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:48.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:48.517+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:48.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:48.517+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:48.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:48 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:48.517+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:48.877 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:48.457+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:48.877 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:48.457+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:48.878 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:48.457+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:49.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:48.852+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:49.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:48.852+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:49.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:48.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:49.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:48.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:49.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:48.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:49.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:48 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:48.943+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:49.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:48.880+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:49.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:48.880+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:49.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:48 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:48.880+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:49.496 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93061]: 2024-01-25 16:22:49.113125889 +0000 UTC m=+0.849443379 container remove 7c191be84bcd2347d7e87b7ed37fa72a650ee52d18f8e34a92ac99d8bc12c18b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-25T16:22:49.749 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93152]: 2024-01-25 16:22:49.402747539 +0000 UTC m=+0.017731853 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:49.749 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93152]: 2024-01-25 16:22:49.511611015 +0000 UTC m=+0.126595367 container create c929a869dea48f0d5abd148feb6bf5f7f0cbca5f08fa6c95c4aaf2a169669d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:22:49.749 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:49.532+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:49.749 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:49.532+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:49.749 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:49.532+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:49.750 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:49 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:49.532+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:49.880 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:49.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:49.881 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:49.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:49.881 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:49.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:50.058 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93152]: 2024-01-25 16:22:49.889293476 +0000 UTC m=+0.504277786 container init c929a869dea48f0d5abd148feb6bf5f7f0cbca5f08fa6c95c4aaf2a169669d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6-deactivate, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-01-25T16:22:50.059 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93152]: 2024-01-25 16:22:49.892610535 +0000 UTC m=+0.507594853 container start c929a869dea48f0d5abd148feb6bf5f7f0cbca5f08fa6c95c4aaf2a169669d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:22:50.059 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:49 smithi033 podman[93152]: 2024-01-25 16:22:49.955897583 +0000 UTC m=+0.570881904 container attach c929a869dea48f0d5abd148feb6bf5f7f0cbca5f08fa6c95c4aaf2a169669d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-01-25T16:22:50.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:49.884+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:50.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:49.884+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:50.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:49.884+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:50.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:49.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:50.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:49.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:50.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:49.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:50.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:49.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:50.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:49.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:50.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:49.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:50.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:49 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:49.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:50.458 INFO:journalctl@ceph.osd.6.smithi033.stdout:Jan 25 16:22:50 smithi033 podman[93152]: 2024-01-25 16:22:50.1829413 +0000 UTC m=+0.797925609 container died c929a869dea48f0d5abd148feb6bf5f7f0cbca5f08fa6c95c4aaf2a169669d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.31.3) 2024-01-25T16:22:50.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:50.511+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:50.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:50.511+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:50.713 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:50.511+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:50.714 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:50 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:50.511+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:50.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.6.service' 2024-01-25T16:22:50.774 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:50.774 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-01-25T16:22:50.774 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-01-25T16:22:50.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.7 2024-01-25T16:22:50.802 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:50.480+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:50.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:50.480+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:50.803 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:50.480+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:51.074 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:50 smithi076 systemd[1]: Stopping Ceph osd.7 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:51.074 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:50.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:51.074 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:50.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:51.074 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:50 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:50.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:51.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:50 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:50.902+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:51.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:50 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:50.902+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:51.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:50 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:50.902+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:51.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:51.001+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:51.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:51.001+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:51.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:51.001+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:51.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:51.001+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:51.412 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.076+0000 7f91925c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:51.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.076+0000 7f91925c4700 -1 osd.7 115 *** Got signal Terminated *** 2024-01-25T16:22:51.413 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.076+0000 7f91925c4700 -1 osd.7 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:51.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:51.487+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:51.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:51.487+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:51.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:51.487+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:51.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:51 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:51.487+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:51.837 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:51.528+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:51.837 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:51.528+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:51.837 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:51.528+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:52.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:51.861+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:52.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:51.861+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:52.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:51 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:51.861+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:52.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:52.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:52.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:52.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:52.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:52.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:52.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:52.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:52.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:52.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:52.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:51 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:51.836+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:52.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:52 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:52.454+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:52.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:52 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:52.454+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:52.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:52 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:52.454+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:52.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:52 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:52.454+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:52.867 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:52.544+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:52.868 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:52.544+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:52.868 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:52.544+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:53.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:52.871+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:53.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:52.871+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:53.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:52.871+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:53.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:52 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:52.871+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.232145+0000 front 2024-01-25T16:22:29.232119+0000 (oldest deadline 2024-01-25T16:22:52.732262+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:52.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:52.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:52 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:52.899+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:53.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:53.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:53.213 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:53.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:53.214 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:53.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:53.214 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:53.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:53.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:53 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:53.423+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:53.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:53 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:53.423+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:53.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:53 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:53.423+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:53.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:53 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:53.423+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:53.900 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:53.558+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:53.900 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:53.558+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:53.900 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:53.558+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:53.900 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:53.558+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:54.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:53.925+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:54.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:53.925+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:54.127 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:53 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:53.925+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:54.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:54.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:54.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:54.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:54.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:54.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:53.903+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:54.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:53.903+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:54.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:53.903+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:54.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:53.903+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.232145+0000 front 2024-01-25T16:22:29.232119+0000 (oldest deadline 2024-01-25T16:22:52.732262+0000) 2024-01-25T16:22:54.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:54 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:54.408+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:54.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:54 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:54.408+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:54.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:54 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:54.408+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:54.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:54 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:54.408+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:54.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:54.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:54.884 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:54.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:54.885 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:54.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:54.885 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:54.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:55.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:54.888+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:55.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:54.888+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:55.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:54.888+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:55.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:54 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:54.888+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.232145+0000 front 2024-01-25T16:22:29.232119+0000 (oldest deadline 2024-01-25T16:22:52.732262+0000) 2024-01-25T16:22:55.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:54.935+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:55.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:54.935+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:55.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:54.935+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:55.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:54.935+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:22:55.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.970+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:55.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.970+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:55.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.970+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:55.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.970+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:55.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:54 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:54.970+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:55.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:55.398+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:55.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:55.398+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:55.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:55.398+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:55.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:55 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:55.398+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:55.879 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:55.580+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:55.879 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:55.580+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:55.879 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:55.580+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:55.879 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:55.580+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:56.162 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:55.883+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:08.228372+0000 front 2024-01-25T16:22:08.228326+0000 (oldest deadline 2024-01-25T16:22:29.328510+0000) 2024-01-25T16:22:56.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:55.883+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:12.829485+0000 front 2024-01-25T16:22:12.829461+0000 (oldest deadline 2024-01-25T16:22:37.529603+0000) 2024-01-25T16:22:56.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:55.883+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:22.230974+0000 front 2024-01-25T16:22:22.231133+0000 (oldest deadline 2024-01-25T16:22:46.931141+0000) 2024-01-25T16:22:56.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:55 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7[54187]: 2024-01-25T16:22:55.883+0000 7f918b3b8700 -1 osd.7 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.232145+0000 front 2024-01-25T16:22:29.232119+0000 (oldest deadline 2024-01-25T16:22:52.732262+0000) 2024-01-25T16:22:56.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:55.954+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:56.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:55.954+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:56.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:55.954+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:56.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:55.954+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:22:56.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:55.944+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:56.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:55.944+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:56.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:55.944+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:56.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:55.944+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:56.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:55 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:55.944+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:56.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:56 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:56.371+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:56.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:56 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:56.371+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:56.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:56 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:56.371+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:56.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:56 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:56.371+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:56.882 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:56 smithi076 podman[73635]: 2024-01-25 16:22:56.607881569 +0000 UTC m=+5.640405177 container died fd91977a95db03d34ec96dde66292aceb087f7036273aca70f085508719a797c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS) 2024-01-25T16:22:56.883 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:56 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:56.585+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:56.883 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:56 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:56.585+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:56.883 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:56 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:56.585+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:56.883 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:56 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:56.585+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:57.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:56 smithi076 podman[73635]: 2024-01-25 16:22:56.96855127 +0000 UTC m=+6.001074903 container cleanup fd91977a95db03d34ec96dde66292aceb087f7036273aca70f085508719a797c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, io.buildah.version=1.31.3, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux ) 2024-01-25T16:22:57.163 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:56 smithi076 bash[73635]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7 2024-01-25T16:22:57.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:56.986+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:57.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:56.986+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:57.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:56.986+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:57.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:56.986+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:22:57.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:56.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:57.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:56.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:57.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:56.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:57.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:56.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:57.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:56 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:56.966+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:57.548 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73652]: 2024-01-25 16:22:57.25292296 +0000 UTC m=+0.638534449 container remove fd91977a95db03d34ec96dde66292aceb087f7036273aca70f085508719a797c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-01-25T16:22:57.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:57 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:57.342+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:57.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:57 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:57.342+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:57.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:57 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:57.342+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:57.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:57 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:57.342+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:57.827 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:57.552+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:57.827 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:57.552+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:57.827 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:57.552+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:57.827 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:57.552+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:57.827 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73741]: 2024-01-25 16:22:57.488372801 +0000 UTC m=+0.017897750 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:22:57.828 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73741]: 2024-01-25 16:22:57.612563441 +0000 UTC m=+0.142088362 container create 96ec2850194f1e06bb59174df39a4996565021efb66b19be046ebcd350d6da4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:57.828 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73741]: 2024-01-25 16:22:57.831240778 +0000 UTC m=+0.360765713 container init 96ec2850194f1e06bb59174df39a4996565021efb66b19be046ebcd350d6da4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux ) 2024-01-25T16:22:58.109 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73741]: 2024-01-25 16:22:57.834583633 +0000 UTC m=+0.364108568 container start 96ec2850194f1e06bb59174df39a4996565021efb66b19be046ebcd350d6da4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123) 2024-01-25T16:22:58.109 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:57 smithi076 podman[73741]: 2024-01-25 16:22:57.914509609 +0000 UTC m=+0.444034551 container attach 96ec2850194f1e06bb59174df39a4996565021efb66b19be046ebcd350d6da4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:22:58.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:58.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:58.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:58.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:58.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:58.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:58.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:58.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:22:58.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:58.007+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:58.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:58.007+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:58.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:58.007+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:58.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:58.007+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:58.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:58 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:58.007+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:58.379 INFO:journalctl@ceph.osd.7.smithi076.stdout:Jan 25 16:22:58 smithi076 podman[73741]: 2024-01-25 16:22:58.113257896 +0000 UTC m=+0.642782842 container died 96ec2850194f1e06bb59174df39a4996565021efb66b19be046ebcd350d6da4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-7-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-01-25T16:22:58.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:58 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:58.556+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:58.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:58 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:58.556+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:58.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:58 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:58.556+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:58.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:58 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:58.556+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:58.703 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.7.service' 2024-01-25T16:22:58.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:22:58.786 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-01-25T16:22:58.786 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-01-25T16:22:58.786 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.8 2024-01-25T16:22:58.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:58.389+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:58.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:58.389+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:58.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:58.389+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:58.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:58 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:58.389+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:22:59.081 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:59.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:59.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:59.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:59.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:22:59.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:58 smithi152 systemd[1]: Stopping Ceph osd.8 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:22:59.082 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.008+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:22:59.083 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.008+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:22:59.083 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.008+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:22:59.083 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.008+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:22:59.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.084+0000 7f8c797c7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:22:59.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.084+0000 7f8c797c7700 -1 osd.8 115 *** Got signal Terminated *** 2024-01-25T16:22:59.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:22:59 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:22:59.084+0000 7f8c797c7700 -1 osd.8 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:22:59.662 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:59 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:59.554+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:22:59.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:59 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:59.554+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:22:59.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:59 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:59.554+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:22:59.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:22:59 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:22:59.554+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:22:59.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:59 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:59.372+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:22:59.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:59 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:59.372+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:22:59.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:59 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:59.372+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:22:59.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:22:59 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:22:59.372+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:00.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:00.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:23:00.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:00.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:23:00.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:00.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:23:00.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:00.012+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:23:00.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:00.045+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:00.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:00.045+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:00.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:00.045+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:00.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:00.045+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:00.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:00 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:00.045+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:00.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:00.330+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:00.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:00.330+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:00.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:00.330+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:00.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:00 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:00.330+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:00.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:00.569+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:00.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:00.569+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:00.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:00.569+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:00.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:00 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:00.569+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:01.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:01.039+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:23:01.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:01.039+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:23:01.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:01.039+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:23:01.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:01.039+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:23:01.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:01.080+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:01.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:01.080+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:01.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:01.080+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:01.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:01.080+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:01.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:01 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:01.080+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:01.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:01 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:01.300+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:01.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:01 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:01.300+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:01.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:01 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:01.300+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:01.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:01 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:01.300+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:01.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:01 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:01.300+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:01.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:01.613+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:01.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:01.613+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:01.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:01.613+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:01.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:01.613+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:01.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:01.613+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:02.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:02.047+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:23:02.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:02.047+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:23:02.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:02.047+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:23:02.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:02.047+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:23:02.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:02.042+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:02.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:02.042+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:02.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:02.042+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:02.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:02.042+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:02.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:02 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:02.042+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:02.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:02.277+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:02.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:02.277+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:02.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:02.277+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:02.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:02.277+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:02.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:02 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:02.277+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:02.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:02.591+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:02.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:02.591+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:02.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:02.591+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:02.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:02.591+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:02.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:02 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:02.591+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:03.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:03.017+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:23:03.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:03.017+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:23:03.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:03.017+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:23:03.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:03.017+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:23:03.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:03.004+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:03.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:03.004+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:03.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:03.004+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:03.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:03.004+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:03.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:03 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:03.004+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:03.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:03.235+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:03.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:03.235+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:03.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:03.235+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:03.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:03.235+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:03.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:03 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:03.235+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:03.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:03.575+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:03.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:03.575+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:03.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:03.575+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:03.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:03.575+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:03.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:03 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:03.575+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:04.053+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.367831+0000 front 2024-01-25T16:22:07.367792+0000 (oldest deadline 2024-01-25T16:22:32.067839+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:04.053+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.568976+0000 front 2024-01-25T16:22:15.568954+0000 (oldest deadline 2024-01-25T16:22:40.269174+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:04.053+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:23.572205+0000 front 2024-01-25T16:22:23.572020+0000 (oldest deadline 2024-01-25T16:22:48.872258+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:04.053+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:29.373519+0000 front 2024-01-25T16:22:29.373338+0000 (oldest deadline 2024-01-25T16:22:54.673366+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8[54471]: 2024-01-25T16:23:04.053+0000 7f8c725bb700 -1 osd.8 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:39.175629+0000 front 2024-01-25T16:22:39.175688+0000 (oldest deadline 2024-01-25T16:23:03.176242+0000) 2024-01-25T16:23:04.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:04.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:04.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:04.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:04.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:04.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:04 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:04.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:04.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:04.258+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:04.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:04.258+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:04.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:04.258+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:04.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:04.258+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:04.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:04 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:04.258+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:04.707 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 podman[74204]: 2024-01-25 16:23:04.439551814 +0000 UTC m=+5.457922021 container died 05ca1baaf031d7c4697a392b0244c2258232078217b34aba375f0fd6e3fa9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:23:04.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:04.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:04.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:04.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:04.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:04.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:04.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:04.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:04.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:04 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:04.549+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:05.004 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 podman[74204]: 2024-01-25 16:23:04.802139237 +0000 UTC m=+5.820509443 container cleanup 05ca1baaf031d7c4697a392b0244c2258232078217b34aba375f0fd6e3fa9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-01-25T16:23:05.004 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:04 smithi152 bash[74204]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8 2024-01-25T16:23:05.377 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74221]: 2024-01-25 16:23:05.060420096 +0000 UTC m=+0.613777038 container remove 05ca1baaf031d7c4697a392b0244c2258232078217b34aba375f0fd6e3fa9f85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3) 2024-01-25T16:23:05.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:05.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:05.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:05.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:05.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:05.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:05 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:05.006+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:05.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:05.297+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:05.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:05.297+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:05.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:05.297+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:05.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:05.297+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:05.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:05 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:05.297+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:05.687 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.317945158 +0000 UTC m=+0.017275498 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:23:05.687 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.462857887 +0000 UTC m=+0.162188266 container create 690a244c5c57740a13adeed971777355228a9ba6a7d906caf45080407b2c80db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-01-25T16:23:05.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:05.515+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:05.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:05.515+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:05.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:05.515+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:05.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:05.515+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:05.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:05 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:05.515+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:05.974 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.691102764 +0000 UTC m=+0.390433172 container init 690a244c5c57740a13adeed971777355228a9ba6a7d906caf45080407b2c80db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-01-25T16:23:05.974 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.695056081 +0000 UTC m=+0.394386422 container start 690a244c5c57740a13adeed971777355228a9ba6a7d906caf45080407b2c80db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:23:05.974 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.773950255 +0000 UTC m=+0.473280608 container attach 690a244c5c57740a13adeed971777355228a9ba6a7d906caf45080407b2c80db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:23:05.974 INFO:journalctl@ceph.osd.8.smithi152.stdout:Jan 25 16:23:05 smithi152 podman[74312]: 2024-01-25 16:23:05.977418844 +0000 UTC m=+0.676749187 container died 690a244c5c57740a13adeed971777355228a9ba6a7d906caf45080407b2c80db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3) 2024-01-25T16:23:06.256 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:06.256 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:06.256 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:06.256 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:06.256 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:06.257 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.011+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:06.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.8.service' 2024-01-25T16:23:06.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.287+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:06.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.287+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:06.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.287+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:06.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.287+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:06.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.287+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:06.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:06.584 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-01-25T16:23:06.584 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-01-25T16:23:06.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.9 2024-01-25T16:23:06.865 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 systemd[1]: Stopping Ceph osd.9 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:06.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:06.504+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:06.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:06.504+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:06.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:06.504+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:06.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:06.504+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:06.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:06 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:06.504+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:07.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:06.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:07.290 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.868+0000 7f225b5e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:07.290 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.868+0000 7f225b5e2700 -1 osd.9 115 *** Got signal Terminated *** 2024-01-25T16:23:07.290 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:06 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:06.868+0000 7f225b5e2700 -1 osd.9 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:23:07.291 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:07.293+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:07.291 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:07.293+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:07.291 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:07.293+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:07.291 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:07.293+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:07.291 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:07 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:07.293+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:07.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:07.537+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:07.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:07.537+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:07.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:07.537+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:07.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:07.537+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:07.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:07 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:07.537+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:08.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:08 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:08.010+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:08.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:08.270+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:08.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:08.270+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:08.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:08.270+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:08.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:08.270+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:08.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:08 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:08.270+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:08.817 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:08.817 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:08.818 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:08.818 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:08.818 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:08.818 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:08 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:08.516+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.032+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.032+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:09.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:09.033+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:09.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:09 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:09.272+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:09.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:09 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:09.272+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:09.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:09 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:09.272+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:09.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:09 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:09.272+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:09.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:09 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:09.272+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:09.662 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:09.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:09.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:09.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:09.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:09.663 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:09 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:09.489+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:10.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:10.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:10.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:10.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:10.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:10.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:10 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:10.019+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:10.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:10.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:10.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:10.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:10.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:10.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:10 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:10.229+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:44.024770+0000 front 2024-01-25T16:22:44.024791+0000 (oldest deadline 2024-01-25T16:23:09.324769+0000) 2024-01-25T16:23:10.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:10.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:10.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:10.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:10.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:10.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:10 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:10.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:11.278 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:11.278 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:11.279 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:11.279 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:11.279 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:11.279 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.009+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:11.558 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.416650+0000 front 2024-01-25T16:21:55.416471+0000 (oldest deadline 2024-01-25T16:22:21.316743+0000) 2024-01-25T16:23:11.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:05.318945+0000 front 2024-01-25T16:22:05.318993+0000 (oldest deadline 2024-01-25T16:22:30.618881+0000) 2024-01-25T16:23:11.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.920485+0000 front 2024-01-25T16:22:15.920458+0000 (oldest deadline 2024-01-25T16:22:38.820541+0000) 2024-01-25T16:23:11.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.121707+0000 front 2024-01-25T16:22:21.121734+0000 (oldest deadline 2024-01-25T16:22:46.421886+0000) 2024-01-25T16:23:11.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:35.823634+0000 front 2024-01-25T16:22:35.823711+0000 (oldest deadline 2024-01-25T16:23:01.123723+0000) 2024-01-25T16:23:11.559 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:11 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9[66567]: 2024-01-25T16:23:11.256+0000 7f22583de700 -1 osd.9 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:44.024770+0000 front 2024-01-25T16:22:44.024791+0000 (oldest deadline 2024-01-25T16:23:09.324769+0000) 2024-01-25T16:23:11.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:11.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:11.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:11.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:11.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:11.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:11 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:11.486+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:12.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:11 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:11.995+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:12.715 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:12 smithi033 podman[93271]: 2024-01-25 16:23:12.43304146 +0000 UTC m=+5.643784202 container died 5967688b2b06fc0fa6db6bcc1292fa8902bf4d305a7b70c844e1f0eb2428ee91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-01-25T16:23:12.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:12.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:12.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:12.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:12.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:12.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:12 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:12.453+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:13.058 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:12 smithi033 podman[93271]: 2024-01-25 16:23:12.79336418 +0000 UTC m=+6.004106921 container cleanup 5967688b2b06fc0fa6db6bcc1292fa8902bf4d305a7b70c844e1f0eb2428ee91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-01-25T16:23:13.059 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:12 smithi033 bash[93271]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9 2024-01-25T16:23:13.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:13.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:13.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:13.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:13.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:13.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:12 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:12.982+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:13.557 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93289]: 2024-01-25 16:23:13.177930056 +0000 UTC m=+0.749147306 container remove 5967688b2b06fc0fa6db6bcc1292fa8902bf4d305a7b70c844e1f0eb2428ee91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-01-25T16:23:13.808 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93377]: 2024-01-25 16:23:13.561857296 +0000 UTC m=+0.110976044 container create 7664e920f49d2e12ee251a94e2e32a1ebf5600b5b34e0be72dee93b6bdf8f453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, ceph=True, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:23:13.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93377]: 2024-01-25 16:23:13.468553749 +0000 UTC m=+0.017672469 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:23:13.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93377]: 2024-01-25 16:23:13.764278583 +0000 UTC m=+0.313397291 container init 7664e920f49d2e12ee251a94e2e32a1ebf5600b5b34e0be72dee93b6bdf8f453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-01-25T16:23:13.809 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93377]: 2024-01-25 16:23:13.767514631 +0000 UTC m=+0.316633342 container start 7664e920f49d2e12ee251a94e2e32a1ebf5600b5b34e0be72dee93b6bdf8f453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=) 2024-01-25T16:23:13.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:13.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:13.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:13.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:13.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:13.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:13 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:13.481+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:14.076 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:13 smithi033 podman[93377]: 2024-01-25 16:23:13.822508921 +0000 UTC m=+0.371627641 container attach 7664e920f49d2e12ee251a94e2e32a1ebf5600b5b34e0be72dee93b6bdf8f453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, ceph=True) 2024-01-25T16:23:14.077 INFO:journalctl@ceph.osd.9.smithi033.stdout:Jan 25 16:23:14 smithi033 podman[93377]: 2024-01-25 16:23:14.060415762 +0000 UTC m=+0.609534473 container died 7664e920f49d2e12ee251a94e2e32a1ebf5600b5b34e0be72dee93b6bdf8f453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-9-deactivate, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux ) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:14.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:14 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:14.013+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:14.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.9.service' 2024-01-25T16:23:14.555 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:14.555 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-01-25T16:23:14.555 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-01-25T16:23:14.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.10 2024-01-25T16:23:14.892 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:14.892 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:14.892 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:14.893 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:14.893 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:14.893 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.446+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:14.893 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 systemd[1]: Stopping Ceph osd.10 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:15.162 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.895+0000 7f74ba080700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:15.162 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.895+0000 7f74ba080700 -1 osd.10 115 *** Got signal Terminated *** 2024-01-25T16:23:15.163 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:14 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:14.895+0000 7f74ba080700 -1 osd.10 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:23:15.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:15.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:15.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:15.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:15.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:15.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.005+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:15.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:15.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:15.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:15.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:15.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:15.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:15 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:15.416+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:16.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:16.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:16.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:16.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:16.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:16.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:15 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:15.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:16.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:16.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:16.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:16.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:16.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:16.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:16 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:16.429+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:17.231 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:17.232 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:17.232 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:17.232 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:17.232 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:17.232 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:16 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:16.963+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:17.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:17.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:17.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:17.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:17.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:17.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:17 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:17.464+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:18.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:17.942+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:18.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:18.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:18.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:18.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:18.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:18.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:18 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:18.499+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:19.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:18 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:18.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:19.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.513+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:59.747071+0000 front 2024-01-25T16:21:59.747233+0000 (oldest deadline 2024-01-25T16:22:21.447153+0000) 2024-01-25T16:23:19.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.513+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:15.549848+0000 front 2024-01-25T16:22:15.549740+0000 (oldest deadline 2024-01-25T16:22:37.750343+0000) 2024-01-25T16:23:19.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.513+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:19.451256+0000 front 2024-01-25T16:22:19.451418+0000 (oldest deadline 2024-01-25T16:22:45.351377+0000) 2024-01-25T16:23:19.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.513+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:31.153223+0000 front 2024-01-25T16:22:31.153375+0000 (oldest deadline 2024-01-25T16:22:52.853342+0000) 2024-01-25T16:23:19.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.514+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.955498+0000 front 2024-01-25T16:22:37.955572+0000 (oldest deadline 2024-01-25T16:23:01.455468+0000) 2024-01-25T16:23:19.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:19 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10[61157]: 2024-01-25T16:23:19.514+0000 7f74b6e7c700 -1 osd.10 115 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-01-25T16:22:46.156645+0000 front 2024-01-25T16:22:46.156628+0000 (oldest deadline 2024-01-25T16:23:08.456628+0000) 2024-01-25T16:23:20.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:20.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:19 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:19.972+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:20.592 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:20 smithi076 podman[73857]: 2024-01-25 16:23:20.344154489 +0000 UTC m=+5.588058298 container died c207b4d9172aee1d7dce6c1dbf42398d934cf9fa3087cd99b61b62d88285f975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-01-25T16:23:20.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:20 smithi076 podman[73857]: 2024-01-25 16:23:20.661563954 +0000 UTC m=+5.905467724 container cleanup c207b4d9172aee1d7dce6c1dbf42398d934cf9fa3087cd99b61b62d88285f975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:23:20.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:20 smithi076 bash[73857]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10 2024-01-25T16:23:20.913 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:20 smithi076 podman[73874]: 2024-01-25 16:23:20.845956813 +0000 UTC m=+0.498568977 container remove c207b4d9172aee1d7dce6c1dbf42398d934cf9fa3087cd99b61b62d88285f975 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:23:21.376 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:21.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.012+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:21.536 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.263441412 +0000 UTC m=+0.103888413 container create 721c28b9a8c1e23c1a2f4fd50d77eb75517b594bfd9d3c41ad2eb09bce6c5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-01-25T16:23:21.537 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.177189031 +0000 UTC m=+0.017636023 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:23:21.537 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.46568915 +0000 UTC m=+0.306136139 container init 721c28b9a8c1e23c1a2f4fd50d77eb75517b594bfd9d3c41ad2eb09bce6c5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-01-25T16:23:21.537 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.468825941 +0000 UTC m=+0.309272917 container start 721c28b9a8c1e23c1a2f4fd50d77eb75517b594bfd9d3c41ad2eb09bce6c5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:23:21.537 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.540402447 +0000 UTC m=+0.380849431 container attach 721c28b9a8c1e23c1a2f4fd50d77eb75517b594bfd9d3c41ad2eb09bce6c5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-01-25T16:23:21.912 INFO:journalctl@ceph.osd.10.smithi076.stdout:Jan 25 16:23:21 smithi076 podman[73968]: 2024-01-25 16:23:21.753279821 +0000 UTC m=+0.593726816 container died 721c28b9a8c1e23c1a2f4fd50d77eb75517b594bfd9d3c41ad2eb09bce6c5a2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:23:22.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:22.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:22.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:22.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:22.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:22.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:22.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:21 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:21.986+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:22.481 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.10.service' 2024-01-25T16:23:22.524 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:22.524 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-01-25T16:23:22.524 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-01-25T16:23:22.525 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.11 2024-01-25T16:23:22.835 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 systemd[1]: Stopping Ceph osd.11 for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.838+0000 7f067306e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.838+0000 7f067306e700 -1 osd.11 115 *** Got signal Terminated *** 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.838+0000 7f067306e700 -1 osd.11 115 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:23.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:23.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:23.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:23.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:23.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:22 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:22.968+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:24.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:24.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:24.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:24.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:24.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:24.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:24.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:23 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:23.925+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:25.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:25.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:25.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:25.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:25.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:25.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:25.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:24 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:24.949+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:26.377 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:26.378 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:25 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:25.911+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-01-25T16:21:55.559257+0000 front 2024-01-25T16:21:55.559279+0000 (oldest deadline 2024-01-25T16:22:20.859459+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6804 osd.1 since back 2024-01-25T16:22:07.062661+0000 front 2024-01-25T16:22:07.062590+0000 (oldest deadline 2024-01-25T16:22:32.362411+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6804 osd.2 since back 2024-01-25T16:22:14.564030+0000 front 2024-01-25T16:22:14.564312+0000 (oldest deadline 2024-01-25T16:22:37.464151+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-01-25T16:22:21.465997+0000 front 2024-01-25T16:22:21.465826+0000 (oldest deadline 2024-01-25T16:22:46.765965+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6812 osd.4 since back 2024-01-25T16:22:26.766600+0000 front 2024-01-25T16:22:26.766473+0000 (oldest deadline 2024-01-25T16:22:52.666545+0000) 2024-01-25T16:23:27.127 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.152:6812 osd.5 since back 2024-01-25T16:22:37.367294+0000 front 2024-01-25T16:22:37.367371+0000 (oldest deadline 2024-01-25T16:23:03.267272+0000) 2024-01-25T16:23:27.128 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:26 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11[61124]: 2024-01-25T16:23:26.872+0000 7f066fe6a700 -1 osd.11 115 heartbeat_check: no reply from 172.21.15.76:6820 osd.7 since back 2024-01-25T16:22:54.369164+0000 front 2024-01-25T16:22:54.369183+0000 (oldest deadline 2024-01-25T16:23:19.668960+0000) 2024-01-25T16:23:28.593 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:28 smithi152 podman[74427]: 2024-01-25 16:23:28.341547764 +0000 UTC m=+5.609731504 container died 7c7e3631730c548360d0ef3ea5e9553037ceca4922b3a13c25491c2d354a7f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-01-25T16:23:28.876 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:28 smithi152 podman[74427]: 2024-01-25 16:23:28.688362948 +0000 UTC m=+5.956546686 container cleanup 7c7e3631730c548360d0ef3ea5e9553037ceca4922b3a13c25491c2d354a7f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2) 2024-01-25T16:23:28.877 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:28 smithi152 bash[74427]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11 2024-01-25T16:23:29.294 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:28 smithi152 podman[74446]: 2024-01-25 16:23:28.98124218 +0000 UTC m=+0.639085449 container remove 7c7e3631730c548360d0ef3ea5e9553037ceca4922b3a13c25491c2d354a7f28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.schema-version=1.0) 2024-01-25T16:23:29.588 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.298565812 +0000 UTC m=+0.114401873 container create 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-01-25T16:23:29.589 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.201656426 +0000 UTC m=+0.017492458 image pull 0b5dc47a38d7bdf336755caaa9af4764ad427a7d8ff345169ca31ad704c9d775 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06 2024-01-25T16:23:29.589 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.509777482 +0000 UTC m=+0.325613524 container init 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-01-25T16:23:29.589 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.51308788 +0000 UTC m=+0.328923896 container start 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, ceph=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-01-25T16:23:29.589 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.592594209 +0000 UTC m=+0.408430246 container attach 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-01-25T16:23:29.876 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:29 smithi152 podman[74539]: 2024-01-25 16:23:29.795278726 +0000 UTC m=+0.611114768 container died 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, io.buildah.version=1.31.3) 2024-01-25T16:23:30.325 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.11.service' 2024-01-25T16:23:30.350 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:30 smithi152 podman[74565]: 2024-01-25 16:23:30.304564364 +0000 UTC m=+0.501532015 container remove 80a31be12f7a234d1e13d5236d56e6903fe2892b12743dcf74ddc7a7a9ce0350 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-01-25T16:23:30.350 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:30 smithi152 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.11.service: Deactivated successfully. 2024-01-25T16:23:30.350 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:30 smithi152 systemd[1]: Stopped Ceph osd.11 for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:23:30.350 INFO:journalctl@ceph.osd.11.smithi152.stdout:Jan 25 16:23:30 smithi152 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@osd.11.service: Consumed 1min 1.072s CPU time. 2024-01-25T16:23:30.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:30.365 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-01-25T16:23:30.366 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-01-25T16:23:30.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.a 2024-01-25T16:23:30.690 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:30 smithi033 systemd[1]: Stopping Ceph mds.a for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:31.058 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:30 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a[73265]: 2024-01-25T16:23:30.693+0000 7f51b03ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:31.058 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:30 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a[73265]: 2024-01-25T16:23:30.693+0000 7f51b03ed700 -1 mds.a *** got signal Terminated *** 2024-01-25T16:23:35.022 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:34 smithi033 podman[93498]: 2024-01-25 16:23:34.704121294 +0000 UTC m=+4.127229372 container died 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123) 2024-01-25T16:23:35.022 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:35 smithi033 podman[93498]: 2024-01-25 16:23:35.026468977 +0000 UTC m=+4.449577053 container cleanup 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux ) 2024-01-25T16:23:35.308 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:35 smithi033 bash[93498]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a 2024-01-25T16:23:35.308 INFO:journalctl@ceph.mds.a.smithi033.stdout:Jan 25 16:23:35 smithi033 podman[93512]: 2024-01-25 16:23:35.202554095 +0000 UTC m=+0.497441933 container remove 71596c75ea09bb3fe732f4db38f64970242a69b4c6f687bd725d48c3e21fefc7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-a, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240123, GIT_CLEAN=True, io.buildah.version=1.31.3) 2024-01-25T16:23:35.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.a.service' 2024-01-25T16:23:35.501 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:35.502 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-01-25T16:23:35.502 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-01-25T16:23:35.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.d 2024-01-25T16:23:35.808 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:35 smithi033 systemd[1]: Stopping Ceph mds.d for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:36.308 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d[72309]: 2024-01-25T16:23:35.905+0000 7fa9547c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:36.308 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:35 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d[72309]: 2024-01-25T16:23:35.905+0000 7fa9547c5700 -1 mds.d *** got signal Terminated *** 2024-01-25T16:23:39.808 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:39 smithi033 podman[93620]: 2024-01-25 16:23:39.545357561 +0000 UTC m=+3.814157744 container died 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3) 2024-01-25T16:23:40.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:39 smithi033 podman[93620]: 2024-01-25 16:23:39.905074602 +0000 UTC m=+4.173874785 container cleanup 1e52c9ce1b9d24d207ae19481653f7bddccb78728e6749e12cc587bd8385a07c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:23:40.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Jan 25 16:23:39 smithi033 bash[93620]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-d 2024-01-25T16:23:40.324 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.d.service' 2024-01-25T16:23:40.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:40.361 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-01-25T16:23:40.361 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-01-25T16:23:40.361 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.g 2024-01-25T16:23:40.808 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:40 smithi033 systemd[1]: Stopping Ceph mds.g for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:40.808 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g[72905]: 2024-01-25T16:23:40.674+0000 7f2d5204a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:40.808 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:40 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g[72905]: 2024-01-25T16:23:40.674+0000 7f2d5204a700 -1 mds.g *** got signal Terminated *** 2024-01-25T16:23:44.680 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:44 smithi033 podman[93741]: 2024-01-25 16:23:44.407952832 +0000 UTC m=+3.828296502 container died cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-01-25T16:23:45.029 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:44 smithi033 podman[93741]: 2024-01-25 16:23:44.766318309 +0000 UTC m=+4.186661980 container cleanup cf895c949215a2bb29970f411cada64e44ee4933a1daf1b1e6dcc4b101c2deaf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3) 2024-01-25T16:23:45.029 INFO:journalctl@ceph.mds.g.smithi033.stdout:Jan 25 16:23:44 smithi033 bash[93741]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-g 2024-01-25T16:23:45.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.g.service' 2024-01-25T16:23:45.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:45.217 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-01-25T16:23:45.217 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-01-25T16:23:45.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.j 2024-01-25T16:23:45.549 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:45 smithi033 systemd[1]: Stopping Ceph mds.j for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:45.807 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j[71959]: 2024-01-25T16:23:45.553+0000 7f2693ebf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:45.808 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:45 smithi033 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j[71959]: 2024-01-25T16:23:45.553+0000 7f2693ebf700 -1 mds.j *** got signal Terminated *** 2024-01-25T16:23:49.048 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:48 smithi033 podman[93864]: 2024-01-25 16:23:48.763998547 +0000 UTC m=+3.323847806 container died 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, ceph=True, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-01-25T16:23:49.308 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:49 smithi033 podman[93864]: 2024-01-25 16:23:49.052909537 +0000 UTC m=+3.612758788 container cleanup 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-01-25T16:23:49.308 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:49 smithi033 bash[93864]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j 2024-01-25T16:23:49.308 INFO:journalctl@ceph.mds.j.smithi033.stdout:Jan 25 16:23:49 smithi033 podman[93878]: 2024-01-25 16:23:49.279180383 +0000 UTC m=+0.515825165 container remove 369c7acc46fa3b898adbd5cdf2b3556ece3c4925bc5a5bc499fed31a4a041dd8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:23:49.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.j.service' 2024-01-25T16:23:49.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:49.434 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-01-25T16:23:49.434 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-01-25T16:23:49.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.b 2024-01-25T16:23:49.807 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:49 smithi076 systemd[1]: Stopping Ceph mds.b for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:50.072 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: 2024-01-25T16:23:49.809+0000 7f7d5a276700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:50.072 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:49 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b[66592]: 2024-01-25T16:23:49.809+0000 7f7d5a276700 -1 mds.b *** got signal Terminated *** 2024-01-25T16:23:52.507 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:52 smithi076 podman[74074]: 2024-01-25 16:23:52.256468328 +0000 UTC m=+2.620540524 container died 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:23:52.782 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:52 smithi076 podman[74074]: 2024-01-25 16:23:52.593711081 +0000 UTC m=+2.957783271 container cleanup 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-01-25T16:23:52.782 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:52 smithi076 bash[74074]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b 2024-01-25T16:23:53.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.b.service' 2024-01-25T16:23:53.034 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:52 smithi076 podman[74100]: 2024-01-25 16:23:52.786475526 +0000 UTC m=+0.528748985 container remove 6ceb75aa7c35cb7c677191dff89e25e0cbf3b67ab3f4811943ae0f4c7b616c7f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-b, ceph=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.31.3, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:23:53.034 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:53 smithi076 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.b.service: Deactivated successfully. 2024-01-25T16:23:53.034 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:53 smithi076 systemd[1]: Stopped Ceph mds.b for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:23:53.035 INFO:journalctl@ceph.mds.b.smithi076.stdout:Jan 25 16:23:53 smithi076 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.b.service: Consumed 10min 28.416s CPU time. 2024-01-25T16:23:53.048 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:53.049 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-01-25T16:23:53.049 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-01-25T16:23:53.049 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.e 2024-01-25T16:23:53.360 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:53 smithi076 systemd[1]: Stopping Ceph mds.e for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:53.662 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: 2024-01-25T16:23:53.363+0000 7f823b183700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:53.662 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:53 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e[65961]: 2024-01-25T16:23:53.363+0000 7f823b183700 -1 mds.e *** got signal Terminated *** 2024-01-25T16:23:56.528 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:56 smithi076 podman[74209]: 2024-01-25 16:23:56.255877599 +0000 UTC m=+2.989141556 container died fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-01-25T16:23:56.869 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:56 smithi076 podman[74209]: 2024-01-25 16:23:56.61442174 +0000 UTC m=+3.347685697 container cleanup fd41a3180d5ce68e9d78b0b0e53854e7686f4bb6374c371a4159e65c9d214de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.31.3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:23:56.869 INFO:journalctl@ceph.mds.e.smithi076.stdout:Jan 25 16:23:56 smithi076 bash[74209]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-e 2024-01-25T16:23:57.036 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.e.service' 2024-01-25T16:23:57.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:23:57.073 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-01-25T16:23:57.073 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-01-25T16:23:57.073 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.h 2024-01-25T16:23:57.397 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:23:57 smithi076 systemd[1]: Stopping Ceph mds.h for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:23:57.662 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:23:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: 2024-01-25T16:23:57.400+0000 7f8b3d9ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:23:57.662 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:23:57 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h[66270]: 2024-01-25T16:23:57.400+0000 7f8b3d9ab700 -1 mds.h *** got signal Terminated *** 2024-01-25T16:24:00.491 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 podman[74330]: 2024-01-25 16:24:00.239406634 +0000 UTC m=+2.947074463 container died 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240123, ceph=True, org.label-schema.vendor=CentOS) 2024-01-25T16:24:00.747 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 podman[74330]: 2024-01-25 16:24:00.575821928 +0000 UTC m=+3.283489766 container cleanup 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.31.3, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:24:00.748 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 bash[74330]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h 2024-01-25T16:24:00.979 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.h.service' 2024-01-25T16:24:01.003 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 podman[74345]: 2024-01-25 16:24:00.751988346 +0000 UTC m=+0.507341440 container remove 08e91dc074576b611818e49fb12f981b0b37b279e2e9feb0d9f3656a226008fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, io.buildah.version=1.31.3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240123, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-01-25T16:24:01.003 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.h.service: Deactivated successfully. 2024-01-25T16:24:01.004 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 systemd[1]: Stopped Ceph mds.h for 55cf2542-bb9a-11ee-95b2-87774f69a715. 2024-01-25T16:24:01.004 INFO:journalctl@ceph.mds.h.smithi076.stdout:Jan 25 16:24:00 smithi076 systemd[1]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.h.service: Consumed 1min 54.216s CPU time. 2024-01-25T16:24:01.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:01.016 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-01-25T16:24:01.016 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-01-25T16:24:01.016 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.k 2024-01-25T16:24:01.325 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:01 smithi076 systemd[1]: Stopping Ceph mds.k for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:24:01.662 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k[65644]: 2024-01-25T16:24:01.328+0000 7f574248f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:24:01.662 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:01 smithi076 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k[65644]: 2024-01-25T16:24:01.328+0000 7f574248f700 -1 mds.k *** got signal Terminated *** 2024-01-25T16:24:05.662 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:05 smithi076 podman[74451]: 2024-01-25 16:24:05.337885554 +0000 UTC m=+4.112940869 container died 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3) 2024-01-25T16:24:06.100 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:05 smithi076 podman[74451]: 2024-01-25 16:24:05.854426032 +0000 UTC m=+4.629481348 container cleanup 70ede9c7e35fb8cf468334b7a907b95c773c002567f4e557dfabf96f85fc8821 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k, io.buildah.version=1.31.3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-01-25T16:24:06.100 INFO:journalctl@ceph.mds.k.smithi076.stdout:Jan 25 16:24:05 smithi076 bash[74451]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-k 2024-01-25T16:24:06.206 DEBUG:teuthology.orchestra.run.smithi076:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.k.service' 2024-01-25T16:24:06.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:06.241 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-01-25T16:24:06.241 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-01-25T16:24:06.241 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.c 2024-01-25T16:24:06.608 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:06 smithi152 systemd[1]: Stopping Ceph mds.c for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:24:06.865 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: 2024-01-25T16:24:06.611+0000 7f86b0e38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:24:06.865 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:06 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c[66598]: 2024-01-25T16:24:06.611+0000 7f86b0e38700 -1 mds.c *** got signal Terminated *** 2024-01-25T16:24:08.566 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:08 smithi152 podman[74645]: 2024-01-25 16:24:08.250438485 +0000 UTC m=+1.802477610 container died 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.31.3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-01-25T16:24:08.850 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:08 smithi152 podman[74645]: 2024-01-25 16:24:08.662055404 +0000 UTC m=+2.214094529 container cleanup 786824a515c1d124d8e45dc5901e3dfa486e9fd8d47c002b0ebab6e30c59ce0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.31.3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.build-date=20240123, GIT_CLEAN=True, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:24:08.850 INFO:journalctl@ceph.mds.c.smithi152.stdout:Jan 25 16:24:08 smithi152 bash[74645]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-c 2024-01-25T16:24:09.073 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.c.service' 2024-01-25T16:24:09.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:09.111 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-01-25T16:24:09.111 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-01-25T16:24:09.111 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.f 2024-01-25T16:24:09.376 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:09 smithi152 systemd[1]: Stopping Ceph mds.f for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:24:09.876 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: 2024-01-25T16:24:09.474+0000 7fa32fc4a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:24:09.876 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:09 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f[66294]: 2024-01-25T16:24:09.474+0000 7fa32fc4a700 -1 mds.f *** got signal Terminated *** 2024-01-25T16:24:12.561 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:12 smithi152 podman[74778]: 2024-01-25 16:24:12.242098959 +0000 UTC m=+2.901266398 container died 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.31.3, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240123, CEPH_POINT_RELEASE=, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183) 2024-01-25T16:24:12.561 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:12 smithi152 podman[74778]: 2024-01-25 16:24:12.566299837 +0000 UTC m=+3.225467280 container cleanup 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.31.3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:24:12.876 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:12 smithi152 bash[74778]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f 2024-01-25T16:24:12.876 INFO:journalctl@ceph.mds.f.smithi152.stdout:Jan 25 16:24:12 smithi152 podman[74790]: 2024-01-25 16:24:12.759153212 +0000 UTC m=+0.513717548 container remove 3b309ea37af66615bd5d4a995f19e127a5d7d692b715ac8d360bbd3b600f041c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.build-date=20240123, ceph=True, io.buildah.version=1.31.3, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240125.044502-661946e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-01-25T16:24:12.987 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.f.service' 2024-01-25T16:24:13.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:13.022 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-01-25T16:24:13.022 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-01-25T16:24:13.023 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.i 2024-01-25T16:24:13.332 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:13 smithi152 systemd[1]: Stopping Ceph mds.i for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:24:13.626 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:13 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: 2024-01-25T16:24:13.336+0000 7ff7f374b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:24:13.626 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:13 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i[66918]: 2024-01-25T16:24:13.336+0000 7ff7f374b700 -1 mds.i *** got signal Terminated *** 2024-01-25T16:24:16.507 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:16 smithi152 podman[74899]: 2024-01-25 16:24:16.247096148 +0000 UTC m=+3.004975767 container died b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, CEPH_POINT_RELEASE=, io.buildah.version=1.31.3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240123, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:24:16.858 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:16 smithi152 podman[74899]: 2024-01-25 16:24:16.603774888 +0000 UTC m=+3.361654499 container cleanup b1f77793b5a29303334289da30b9c09449425b9e1177793fa2ccfa5caaaf7490 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-01-25T16:24:16.858 INFO:journalctl@ceph.mds.i.smithi152.stdout:Jan 25 16:24:16 smithi152 bash[74899]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-i 2024-01-25T16:24:17.015 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.i.service' 2024-01-25T16:24:17.049 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:17.049 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-01-25T16:24:17.050 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-01-25T16:24:17.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.l 2024-01-25T16:24:17.369 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:17 smithi152 systemd[1]: Stopping Ceph mds.l for 55cf2542-bb9a-11ee-95b2-87774f69a715... 2024-01-25T16:24:17.626 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l[65981]: 2024-01-25T16:24:17.373+0000 7f1f3ad09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-01-25T16:24:17.626 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:17 smithi152 ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l[65981]: 2024-01-25T16:24:17.373+0000 7f1f3ad09700 -1 mds.l *** got signal Terminated *** 2024-01-25T16:24:19.502 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:19 smithi152 podman[75018]: 2024-01-25 16:24:19.196466214 +0000 UTC m=+1.930496948 container died 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240123, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.31.3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240125.044502-661946e) 2024-01-25T16:24:19.503 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:19 smithi152 podman[75018]: 2024-01-25 16:24:19.507584372 +0000 UTC m=+2.241615108 container cleanup 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240123, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-01-25T16:24:19.803 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:19 smithi152 bash[75018]: ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l 2024-01-25T16:24:19.803 INFO:journalctl@ceph.mds.l.smithi152.stdout:Jan 25 16:24:19 smithi152 podman[75032]: 2024-01-25 16:24:19.700334643 +0000 UTC m=+0.501893059 container remove 81d08ae07be946653fdb43b146bf7d7ed1070d5998ce3f1e0db41fb6e94f398b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5605ea5d84bbf2ac5d8e2f7c50aa193735a361c5be4d9635ebaf6df822105f06, name=ceph-55cf2542-bb9a-11ee-95b2-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240123, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=60f64185d6ee4a1620a4e475ec29fb91e0b44183, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240125.044502-661946e, io.buildah.version=1.31.3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-01-25T16:24:19.830 DEBUG:teuthology.orchestra.run.smithi152:> sudo pkill -f 'journalctl -f -n 0 -u ceph-55cf2542-bb9a-11ee-95b2-87774f69a715@mds.l.service' 2024-01-25T16:24:19.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-01-25T16:24:19.864 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-01-25T16:24:19.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force --keep-logs 2024-01-25T16:24:20.061 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:24:22.480 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force --keep-logs 2024-01-25T16:24:22.670 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:24:25.323 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force --keep-logs 2024-01-25T16:24:25.513 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:24:27.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:24:27.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:24:28.022 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-01-25T16:24:28.051 INFO:tasks.cephadm:Archiving crash dumps... 2024-01-25T16:24:28.053 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi033/crash 2024-01-25T16:24:28.054 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash -- . 2024-01-25T16:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash: Cannot open: No such file or directory 2024-01-25T16:24:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-01-25T16:24:28.100 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi076/crash 2024-01-25T16:24:28.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash -- . 2024-01-25T16:24:28.145 INFO:teuthology.orchestra.run.smithi076.stderr:tar: /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash: Cannot open: No such file or directory 2024-01-25T16:24:28.146 INFO:teuthology.orchestra.run.smithi076.stderr:tar: Error is not recoverable: exiting now 2024-01-25T16:24:28.148 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi152/crash 2024-01-25T16:24:28.148 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash -- . 2024-01-25T16:24:28.185 INFO:teuthology.orchestra.run.smithi152.stderr:tar: /var/lib/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/crash: Cannot open: No such file or directory 2024-01-25T16:24:28.185 INFO:teuthology.orchestra.run.smithi152.stderr:tar: Error is not recoverable: exiting now 2024-01-25T16:24:28.186 INFO:tasks.cephadm:Checking cluster log for badness... 2024-01-25T16:24:28.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-01-25T16:24:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:grep: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph.log: No such file or directory 2024-01-25T16:24:28.222 INFO:tasks.cephadm:Compressing logs... 2024-01-25T16:24:28.222 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-25T16:24:28.265 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-25T16:24:28.268 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-25T16:24:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-25T16:24:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.6.log 2024-01-25T16:24:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.a.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose --/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.6.log: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.g.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.j.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.3.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose --/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.j.log: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.x.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.0.log 2024-01-25T16:24:28.294 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-25T16:24:28.295 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.9.log 2024-01-25T16:24:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.l.log 2024-01-25T16:24:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:: No such file or directory 2024-01-25T16:24:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.i.log 2024-01-25T16:24:28.296 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.11.log 2024-01-25T16:24:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.l.log: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.z.log 2024-01-25T16:24:28.297 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.8.log 2024-01-25T16:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log: 94.6% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.g.log.gz 2024-01-25T16:24:28.299 INFO:teuthology.orchestra.run.smithi033.stderr: 94.7% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.j.log.gz 2024-01-25T16:24:28.301 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.5.log 2024-01-25T16:24:28.301 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.z.log: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.f.log 2024-01-25T16:24:28.301 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.2.log 2024-01-25T16:24:28.301 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.c.log 2024-01-25T16:24:28.302 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.2.log: 94.7% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.l.log.gz 2024-01-25T16:24:28.302 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-25T16:24:28.304 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.c.log: 92.1% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.z.log.gz 2024-01-25T16:24:28.304 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.c.log 2024-01-25T16:24:28.306 INFO:teuthology.orchestra.run.smithi076.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-01-25T16:24:28.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.10.log 2024-01-25T16:24:28.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.b.log 2024-01-25T16:24:28.307 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.4.log 2024-01-25T16:24:28.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.10.log: /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.1.log 2024-01-25T16:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.d.log 2024-01-25T16:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.a.log 2024-01-25T16:24:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.d.log: 91.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-25T16:24:28.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.k.log 2024-01-25T16:24:28.311 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.h.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.7.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.e.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.y.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.b.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-25T16:24:28.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.b.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.k.log.gz 2024-01-25T16:24:28.313 INFO:teuthology.orchestra.run.smithi033.stderr: 94.5% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.d.log.gz 2024-01-25T16:24:28.313 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-25T16:24:28.315 INFO:teuthology.orchestra.run.smithi076.stderr: 92.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.y.log.gz 2024-01-25T16:24:28.317 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-01-25T16:24:28.319 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.c.log: 90.9% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-25T16:24:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.a.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.a.log.gz 2024-01-25T16:24:28.322 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-25T16:24:28.337 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-25T16:24:28.371 INFO:teuthology.orchestra.run.smithi152.stderr: 94.6% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.f.log.gz 2024-01-25T16:24:28.474 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-volume.log.gz 2024-01-25T16:24:28.521 INFO:teuthology.orchestra.run.smithi076.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-01-25T16:24:28.659 INFO:teuthology.orchestra.run.smithi033.stderr: 89.9% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mgr.x.log.gz 2024-01-25T16:24:28.845 INFO:teuthology.orchestra.run.smithi152.stderr: 91.9% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.c.log.gz 2024-01-25T16:24:29.236 INFO:teuthology.orchestra.run.smithi033.stderr: 90.4% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.a.log.gz 2024-01-25T16:24:30.507 INFO:teuthology.orchestra.run.smithi076.stderr: 92.9% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mon.b.log.gz 2024-01-25T16:24:32.144 INFO:teuthology.orchestra.run.smithi076.stderr: 95.2% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.e.log.gz 2024-01-25T16:24:32.603 INFO:teuthology.orchestra.run.smithi033.stderr: 94.2% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.9.log.gz 2024-01-25T16:24:33.795 INFO:teuthology.orchestra.run.smithi152.stderr: 94.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.11.log.gz 2024-01-25T16:24:34.625 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.6.log.gz 2024-01-25T16:24:34.783 INFO:teuthology.orchestra.run.smithi033.stderr: 93.7% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.0.log.gz 2024-01-25T16:24:35.065 INFO:teuthology.orchestra.run.smithi152.stderr: 95.5% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.c.log.gz 2024-01-25T16:24:35.818 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.3.log.gz 2024-01-25T16:24:35.820 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-01-25T16:24:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m7.542s 2024-01-25T16:24:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m25.102s 2024-01-25T16:24:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.887s 2024-01-25T16:24:36.379 INFO:teuthology.orchestra.run.smithi152.stderr: 96.3% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.i.log.gz 2024-01-25T16:24:36.528 INFO:teuthology.orchestra.run.smithi152.stderr: 94.1% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.2.log.gz 2024-01-25T16:24:37.492 INFO:teuthology.orchestra.run.smithi152.stderr: 94.2% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.5.log.gz 2024-01-25T16:24:37.668 INFO:teuthology.orchestra.run.smithi152.stderr: 94.4% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.8.log.gz 2024-01-25T16:24:37.670 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-01-25T16:24:37.670 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m9.389s 2024-01-25T16:24:37.670 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m46.183s 2024-01-25T16:24:37.670 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m1.531s 2024-01-25T16:24:40.677 INFO:teuthology.orchestra.run.smithi076.stderr: 94.0% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.4.log.gz 2024-01-25T16:24:42.959 INFO:teuthology.orchestra.run.smithi076.stderr: 93.8% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.1.log.gz 2024-01-25T16:24:44.202 INFO:teuthology.orchestra.run.smithi076.stderr: 94.2% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.10.log.gz 2024-01-25T16:24:46.152 INFO:teuthology.orchestra.run.smithi076.stderr: 94.2% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-osd.7.log.gz 2024-01-25T16:25:16.490 INFO:teuthology.orchestra.run.smithi076.stderr: 94.5% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.h.log.gz 2024-01-25T16:27:59.226 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/55cf2542-bb9a-11ee-95b2-87774f69a715/ceph-mds.b.log.gz 2024-01-25T16:27:59.229 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-01-25T16:27:59.229 INFO:teuthology.orchestra.run.smithi076.stderr:real 3m30.947s 2024-01-25T16:27:59.229 INFO:teuthology.orchestra.run.smithi076.stderr:user 3m46.633s 2024-01-25T16:27:59.229 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m7.927s 2024-01-25T16:27:59.230 INFO:tasks.cephadm:Archiving logs... 2024-01-25T16:27:59.230 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi033/log 2024-01-25T16:27:59.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-25T16:28:02.757 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi076/log 2024-01-25T16:28:02.758 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-25T16:28:37.491 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi152/log 2024-01-25T16:28:37.493 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-25T16:28:42.598 INFO:tasks.cephadm:Removing cluster... 2024-01-25T16:28:42.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force 2024-01-25T16:28:42.797 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:28:43.092 DEBUG:teuthology.orchestra.run.smithi076:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force 2024-01-25T16:28:43.288 INFO:teuthology.orchestra.run.smithi076.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:28:43.781 DEBUG:teuthology.orchestra.run.smithi152:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 55cf2542-bb9a-11ee-95b2-87774f69a715 --force 2024-01-25T16:28:43.974 INFO:teuthology.orchestra.run.smithi152.stdout:Deleting cluster with fsid: 55cf2542-bb9a-11ee-95b2-87774f69a715 2024-01-25T16:28:44.275 INFO:tasks.cephadm:Removing cephadm ... 2024-01-25T16:28:44.276 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-25T16:28:44.293 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-25T16:28:44.309 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf /home/ubuntu/cephtest/cephadm 2024-01-25T16:28:44.324 INFO:tasks.cephadm:Teardown complete 2024-01-25T16:28:44.325 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-25T16:28:44.337 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-25T16:28:44.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-25T16:28:44.341 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-25T16:28:44.352 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-25T16:28:44.476 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-25T16:28:44.487 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-25T16:28:44.488 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T16:28:44.490 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T16:28:44.493 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-25T16:28:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-01-25T16:28:44.506 INFO:teuthology.orchestra.run.smithi076.stderr:bash: line 1: ntpq: command not found 2024-01-25T16:28:44.506 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-01-25T16:28:44.508 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T16:28:44.508 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-01-25T16:28:44.508 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 -582us[ -546us] +/- 80ms 2024-01-25T16:28:44.508 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1701us[-1665us] +/- 60ms 2024-01-25T16:28:44.509 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +3202us[+3202us] +/- 87ms 2024-01-25T16:28:44.509 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 20 -578us[ -555us] +/- 80ms 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 19 -1694us[-1672us] +/- 60ms 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +3222us[+3244us] +/- 87ms 2024-01-25T16:28:44.511 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T16:28:44.540 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-25T16:28:44.540 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2024-01-25T16:28:44.540 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 21 -587us[ -571us] +/- 80ms 2024-01-25T16:28:44.541 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 17 -1720us[-1704us] +/- 60ms 2024-01-25T16:28:44.541 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 +3186us[+3202us] +/- 87ms 2024-01-25T16:28:44.541 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-25T16:28:44.552 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-25T16:28:44.563 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-25T16:28:44.564 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-25T16:28:44.576 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-25T16:28:44.663 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-25T16:28:44.762 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-25T16:28:44.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T16:28:44.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:44.866 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T16:28:44.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:44.897 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-25T16:28:44.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:44.927 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-25T16:28:44.938 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-25T16:28:44.949 INFO:teuthology.task.internal:Duration was 2296.768124 seconds 2024-01-25T16:28:44.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-25T16:28:44.963 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-25T16:28:44.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-25T16:28:44.966 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-25T16:28:44.968 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-25T16:28:45.010 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T16:28:45.011 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T16:28:45.050 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-25T16:28:45.453 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-25T16:28:45.453 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-01-25T16:28:45.453 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-25T16:28:45.476 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-01-25T16:28:45.476 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-25T16:28:45.505 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-01-25T16:28:45.505 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-25T16:28:45.541 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-25T16:28:45.541 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-25T16:28:45.543 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-25T16:28:45.547 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-25T16:28:45.610 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-25T16:28:45.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-25T16:28:45.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-25T16:28:45.620 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-25T16:28:46.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-25T16:28:46.036 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-25T16:28:46.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-25T16:28:46.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-25T16:28:46.093 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-25T16:28:46.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-25T16:28:46.133 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.136 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.138 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.158 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-01-25T16:28:46.164 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-01-25T16:28:46.186 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-01-25T16:28:46.208 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:46.227 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:46.243 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-25T16:28:46.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-25T16:28:46.262 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-25T16:28:46.275 INFO:teuthology.task.internal:Transferring archived files... 2024-01-25T16:28:46.277 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi033 2024-01-25T16:28:46.277 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-25T16:28:46.321 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi076 2024-01-25T16:28:46.321 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-25T16:28:46.360 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549/remote/smithi152 2024-01-25T16:28:46.360 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-25T16:28:46.399 INFO:teuthology.task.internal:Removing archive directory... 2024-01-25T16:28:46.399 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-25T16:28:46.401 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-25T16:28:46.404 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-25T16:28:46.448 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-25T16:28:46.466 INFO:teuthology.task.internal:Not uploading archives. 2024-01-25T16:28:46.467 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-25T16:28:46.481 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-25T16:28:46.482 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-25T16:28:46.485 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-25T16:28:46.488 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-25T16:28:46.496 INFO:teuthology.orchestra.run.smithi033.stdout: 265291 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 25 16:28 /home/ubuntu/cephtest 2024-01-25T16:28:46.499 INFO:teuthology.orchestra.run.smithi076.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 25 16:28 /home/ubuntu/cephtest 2024-01-25T16:28:46.503 INFO:teuthology.orchestra.run.smithi152.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 25 16:28 /home/ubuntu/cephtest 2024-01-25T16:28:46.504 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-25T16:28:46.518 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-25T16:28:46.560 INFO:teuthology.nuke:Checking targets against current locks 2024-01-25T16:28:46.588 DEBUG:teuthology.nuke:shortname: smithi033 2024-01-25T16:28:46.588 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-25T16:28:46.615 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.415543', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM0hyvbZJ9gvsAeRpvp7buH5ak1m4WEFb7CXwExj0AtpcaGc+4PY3ww6tbD9fsky7IpVAtVf4RU3ssYUskryAaQ='} 2024-01-25T16:28:46.620 DEBUG:teuthology.nuke:shortname: smithi076 2024-01-25T16:28:46.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-25T16:28:46.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.417553', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE3Pqla8kAnrFl5MUmD5SNHJkHK8VKHYPgL7gKNVcDPlOUMoIv55HIIobt+gKHf1QrNDEGole7gbhG/UFLXpUck='} 2024-01-25T16:28:46.648 DEBUG:teuthology.nuke:shortname: smithi152 2024-01-25T16:28:46.649 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-25T16:28:46.670 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-25_13:52:19-fs-wip-vshankar-testing-20240125.044502-testing-default-smithi/7531549', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-25 15:42:02.419315', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI4tanWusQYW277aM8Ws84D77qIL+D1eqc0QTHN2sXmZS9Xyn2hBYpAfK9b3gnHtREHPbe9llcoIFdAineVZGFw='} 2024-01-25T16:28:46.681 INFO:teuthology.orchestra.console.smithi033:Power off 2024-01-25T16:28:46.681 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-25T16:28:46.705 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-01-25T16:28:46.706 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:46.724 INFO:teuthology.orchestra.console.smithi076:Power off 2024-01-25T16:28:46.725 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-25T16:28:46.739 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-01-25T16:28:46.747 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2024-01-25T16:28:46.747 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:46.767 INFO:teuthology.orchestra.console.smithi152:Power off 2024-01-25T16:28:46.767 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-25T16:28:46.777 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-01-25T16:28:46.790 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2024-01-25T16:28:46.790 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:46.813 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-01-25T16:28:50.740 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:50.777 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:50.815 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:50.852 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-01-25T16:28:50.888 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-01-25T16:28:50.925 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-01-25T16:28:54.858 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:54.890 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:54.927 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:54.970 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-01-25T16:28:55.002 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2024-01-25T16:28:55.037 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-01-25T16:28:55.071 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-01-25T16:28:55.138 INFO:teuthology.orchestra.console.smithi152:Power off completed 2024-01-25T16:28:59.004 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-25T16:28:59.114 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2024-01-25T16:28:59.215 INFO:teuthology.orchestra.console.smithi076:Power off completed 2024-01-25T16:28:59.322 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/suites/dbench}} duration: 2296.768124103546 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi033 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=661946e7044723130c03627323f6a879a9848cb5 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=80c39e98449a4b6eaae28d70be38461b status: fail success: false 2024-01-25T16:28:59.323 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-25T16:28:59.411 INFO:teuthology.run:FAIL